When the Results Are Good, Showcase Them!

Good assessment helps us determine where improvements are needed. We know this. It’s basic to good teaching, and it happens all the time in courses. It might mean rephrasing or eliminating exam questions, redesigning a rubric for greater clarity, or implementing active learning strategies to help students learn difficult material.

Continuous improvement is not the sole purpose of assessment, however. Assessment, by definition, is a way to identify weaknesses and strengths. It should, and often does, yield results that might be celebrated and used to tell others about our programs. This aspect of assessment is often overlooked, though. In part, that is because it has been drummed into our heads that assessment is a way to ferret out where our efforts are unsuccessful. Another reason is that humans are hardwired for negative bias. We look at survey results, for example, and tend to see only where students are dissatisfied or unhappy.

Assessments at Utica University, particularly those done by academic departments, have produced celebration-worthy findings that that underscore the value of our programs, and we need to share these results with prospective students, their parents, potential employers, advisory boards, and community partners. They are what distinguishes us.  

In biology, for example, the results earned on the Major Field Tests in molecular biology and genetics, cell biology, organismal biology, and population biology, evolution, and ecology are benchmarked with national institutional means. Utica University’s mean scores were higher than the national mean in each subject area.

Likewise, physics students continue to surpass the national average on a survey tool measuring conceptual knowledge of mechanics.

English majors demonstrate marked improvements in their writing abilities at the end of or close to the end of their academic tenure at the University, evidence of the value-added of this program.

On a standardized pre/post assessment, students in undergraduate business programs and the MBA program demonstrated significant growth in their knowledge and skills, and the mean scores earned on the post assessment were higher than those earned by an Accreditation Council for Business Schools and Programs peer group.   

A student satisfaction survey administered in November 2021 showed high levels of satisfaction with support offerings, namely library resources and services, career services, tutoring, academic advising, and the availability of counseling.

What’s most encouraging about all these findings is that they came from a year when pandemic-weary students showed signs of disengagement and poor motivation.

Assessment leader Linda Suskie writes, “The higher education community has a long-standing culture of keeping its light under the proverbial bushel basket and not sharing the story of its successes with its public audiences.”

It’s time we changed that. It’s time we move the narrative past clichés and use assessment results to showcase the strengths of our programs and our students, as well as to improve our educational effectiveness.  

                                                                        Work Cited

Linda Suskie.  Assessing Student Learning: A Common Sense Guide. 2^nd ed. 

                San Francisco: Jossey- Bass, 2009.   

Standardized Testing: A One-Size-Fits-All Assessment

 By Michelle Boucher

When we first started with formal department-wide assessment in our department, we identified the  skills we want our students to develop as chemists. These are the very skills that the American Chemical Society (ACS) expect students to demonstrate in a certified program. It made sense for us to assess these skills using ACS standardized exams as final examinations in those classes for which ACS offered a standardized exam.

This assessment strategy wasn’t perfect, but the exams offered the potential for a beautiful tool for assessment. They are created by a committee of faculty from diverse institutions, they are checked by the Examinations Institute Board of Trustees, and data on student performance is collected and collated by the ACS and distributed to every school administering the specific exams. Specific means and medians for each question and the exam overall are available from a reasonably large pool of students who took the exam nationwide in a given year. The exams are refreshed every 5-10 years through committee, and fresh student outcome data is collected.

While this all sounds fantastic in theory, the reality of the exams proved less utopian. The committee of faculty from “diverse institutions” who write the exam do not typically include equal representation from schools that serve first generation students or schools that are historically black schools or from community colleges. The faculty represented are typically a mix of faculty from Ph.D. granting institutions, highly selective small liberal arts colleges, and perhaps only 1-2 faculty (out of 17-20) from all the “other types of schools” (community colleges, small comprehensive colleges like ours, etc.). All the exams are political in one form or another. The ACS is clearly dictating, through this exam, what it feels to be of importance in the specific course. For example, one iteration of the organic exam had two questions (out of 70) on green chemistry, when there was a faculty representative on the committee who was a green chemist. The most recent organic exam has 4 carbonyl reactions that are specific subfields and also “named reactions” (after the chemist(s) who discovered or publicized the use of the reactions) in part as a direct answer to a recent push in chemical education to minimize the use of “named reactions” in the interests of promoting a more inclusive classroom experience.

Additionally, there are issues concerning equity and standardized exams. It has been shown, time and again in sociological and pedagogical literature, that there are inherent equity issues around standardized exams. There continues to be discussion and research nationwide around what root causes exist that lead to underserved students (students of color, students from lower economic brackets, first-generation students) and students who identify as women to perform at lower levels on standardized exams. Regardless of the reason, the faculty in our department believe disparities exist. We see it in our incoming students, who benefit greatly from the holistic application review that Utica University offers and are often high achieving students with poor standardized test scores, and we see it on our final standardized exams, where students who have performed exceptionally well all semester choke on a standardized exam.

This past year, we experienced another issue with these standardized exams:  we could not vouch for their reliability. The standardized exam results, in fact all of the assessment data, showed little to no impact on our student education and the 2020-2021 COVID-19 experience. While that makes us go “Yay!”, we know that our students right now have fewer skills than they would typically have at this point in their education. We know that our students are faring better than some cohorts at other schools; we all talk with multiple people at other institutions and are active in ChemEd circles nationally, and we can see where our students place. We know our students have a much smaller “COVID-lag” than cohorts at other institutions. We are proud of that. But we know there is a knowledge and experience gap, and our assessment methods do not show that.

We are making the move away from the ACS standardized exams, or we plan to use them the way we want to use them and bend them to our own wills. There is absolutely no good reason for our program to be dictated to, our learning goals determined and defined, by a committee of homogenous professors protecting a status quo that we have dedicated our professional lives to overthrowing.  In our department, the age of one-size-fits-all assessment and the lies it propagated is over.