The Limits and Usefulness of Anecdotal Evidence in Assessment

 We’ve all been at those meetings where someone steers the conversation with claims such as, “Students don’t like having early morning classes, and when they do, they leave the institution.” There’s no source for this statement; there’s no evidence in any survey, focus group, or analysis of enrollment data to substantiate the claim.

People typically respond to statements such as that one by saying, “That’s anecdotal evidence.” Not so. Unsubstantiated claims don’t rise to the level of anecdotal evidence. They remain exactly what they are: unverified statements, usually intended to promote an agenda or enhance a speaker’s credibility.

Anecdotal evidence refers to stories about people and their experiences.  By itself, an anecdote is not reliable evidence. However, when anecdotes are used judiciously in conjunction with quantitative data, they provide insight into what the data might mean. Further, anecdotes have emotional appeal. They remind us why we should care about the data in the first place.

Student responses to the climate survey data at UC (March 2019) provide excellent examples of how anecdotes may be paired with data to advance understanding.  On the survey, more than half of students of color reported feeling that they did not matter in classes taught by white faculty, and compared to white students, fewer students of color reported feeling affirmed by white faculty.

When students were asked to respond to these data points by narrating their experiences, they described occasions where white faculty directed their questions solely to white students and where white faculty ignored racist comments made by other students in the class. They gave examples of classes where the majority of students performed poorly on an exam and the professors indicated that the reason is because the course material is rigorous, and the students are unprepared for the demands of the subject. They mentioned classes where the instructor never learned their names and seldom acknowledged them outside of class.

On the plus side, they spoke of positive experiences that made them feel affirmed, supported, and part of a community.

These stories—anecdotes—give texture and meaning to data that might perplex, dismay, or be easily dismissed.

The limits of anecdotal evidence, which is usually based on individual experience, should be obvious. Its usefulness, though, cannot be overlooked. One benefit is that invites inquiry and may have implications for research. Its major benefit is described by Michael Shermer, publisher of Skeptic: “Anecdotes  . . . help in explaining data points that do not make sense. Hearing stories about data points that do not make intuitive sense can uncover the hidden variables that are really driving the results.”

Work Cited

Shermer, Michael. "How Anecdotal Evidence Can Undermine Scientific Results: Why Subjective Anecdotes Often Trump Objective Data." Scientific American (2008).