Evidence-Based Storytelling: How Assessment Helps Us Tell a Compelling Narrative

There’s no question that assessment is often regarded as a fill-in-the-blank, paint-by-number bureaucratic enterprise. For too long, assessment specialists and accrediting agencies promoted a linear approach where faculty make specific changes in courses and curriculum based on assessment findings with the aim of improving student learning and then re-assess to document the effects of these changes.  

Yet Natasha Jankowski, former executive director of the National Institute for Learning Outcomes Assessment asks, “Can one ever actually know that the changes made enhanced student learning?”

So much influences a person’s growth and development in the years between starting a degree and earning one. Oral communication skills, for example, might be developed a great deal in an undergraduate curriculum, but so, too, might these abilities be improved by the experiences a student has in the co-curricular environment or the world of work.

There are modifications we can make in an effort to maximize student learning and ensure that all students have the opportunity to develop the knowledge, skills, and competencies faculty consider critical in a discipline. However, we simply cannot make emphatic claims to a causal relationship between what we did and the extent to which it improved student learning.

Jankowski advocates for a different approach to how we might demonstrate educational effectiveness. She argues that the meaning we draw from assessment findings, our understanding of the data, constitutes the important narrative. This meaning shapes the story we derive from the evidence we have gathered. 

“In assessment there is so much doing that there is limited time, if any, built into reflecting upon the data and deciding what it all says, what argument might be made, and what story it tells about students and their learning” (page 11).

Stories give evidence meaning. The 2020-2021 assessment report from the Department of Philosophy documents an important story about teaching and learning in a pandemic where COVID fatigue resulted in students’ failing to complete assignments as well as an increase in cheating. The quantitative findings suggest that student learning was on a downward trend. But the numbers alone don’t tell the story. The meaning inferred from the numbers by the faculty quoted in the report’s narrative does.

The report from the English Department provides an illustration that shows how students achieve learning beyond that which is articulated in a program goal. Students who participate in the design and creation of Ampersand, the College’s literary journal, “go beyond” the goal of making authorial choices: “[T]hey learn to collaborate, they learn skills of layout and editing as they produce a publication that appears in both print and online forms.”

Evidence-based stories—stories informed by the quantitative and qualitative evidence we systematically gather—are how we best illustrate the value and impact of our individual programs and of higher education. These stories also tell us what we need to change or improve in our teaching, course content, and curriculum.

“Some of our stories are tragedies,” Jankowski writes, “and some are tales of heroics and adventures” (page 12). They provide us with a richer, deeper, and more meaningful way to discuss assessment findings than the linear, formulaic approach does. Whether our stories have a happy endings or sad conclusions, they deserve to be told. 

Jankowski, N. (2021, February). Evidence-Based Storytelling in Assessment. Occasional Paper No. 50). Urbana: IL: University         of Illinois and Indiana University. National Institute for Learning Outcomes Assessment.

Capturing the COVID Narrative

The full implications of how COVID-19 will impact higher education probably won’t be realized for another four or five years. Still, there’s much we already know about the pandemic’s effect: lower than expected enrollments, fiscal challenges, and increased disenfranchisement among students are just a few of the immediate consequences.  

The 2020-2021 assessments in co-curricular and student support services provide authentic and compelling evidence of COVID’s effect on Utica College students and operations. In particular, the report from the Department of Athletics offers a narrative about a year when “all of the plans in place . . . had been compromised,” a year of “twists, turns, starts, and stops” for which there was no playbook.

One method used to assess the student-athlete experience is the “Athlete Viewpoint,” an instrument that measures student-athlete commitment and well-being, team culture, academic advising, institutional acceptance, and, in 2020, the impact of COVID.

A significant finding from the department’s assessment was that more than half of student-athletes who responded to the survey indicated challenges with mental health. The report notes that COVID-19 created a “rapidly changing landscape” and “student athletes did not know what roadmap to follow . . .  to successfully start and complete a season.”

In response to this finding, the department plans to provide student-athletes access to more internal and external mental health and general health and wellness resources.

While the 2020-2021 academic year witnessed countless challenges and disappointments for UC’s Department of Athletics, the report also shows that despite these hardships, our student-athletes persisted and, in some instances, thrived. Close to 90% of conference games were completed, 43.75% of eligible teams finished among the top 4 teams in their conference, 31.25% qualified for playoffs, and over a quarter of the teams advanced to the conference championship.

Assessment is more than quantitative evidence gathered for the purpose of compliance. It is a way to use evidence to tell a story. The report from the Department of Athletics is a story of student-athlete resilience, and, I might venture to add, story of historical significance.