By Michelle Boucher
When we first started with formal department-wide assessment
in our department, we identified the skills we want our students to develop as
chemists. These are the very skills that the American Chemical Society (ACS) expect
students to demonstrate in a certified program. It made sense for us to assess
these skills using ACS standardized exams as final examinations in those
classes for which ACS offered a standardized exam.
This assessment strategy wasn’t perfect, but the exams offered
the potential for a beautiful tool for assessment. They are created by a
committee of faculty from diverse institutions, they are checked by the
Examinations Institute Board of Trustees, and data on student performance is
collected and collated by the ACS and distributed to every school administering
the specific exams. Specific means and medians for each question and the exam
overall are available from a reasonably large pool of students who took the exam
nationwide in a given year. The exams are refreshed every 5-10 years through
committee, and fresh student outcome data is collected.
While this all sounds fantastic in theory, the reality of
the exams proved less utopian. The committee of faculty from “diverse
institutions” who write the exam do not typically include equal representation
from schools that serve first generation students or schools that are
historically black schools or from community colleges. The faculty represented
are typically a mix of faculty from Ph.D. granting institutions, highly
selective small liberal arts colleges, and perhaps only 1-2 faculty (out of
17-20) from all the “other types of schools” (community colleges, small
comprehensive colleges like ours, etc.). All the exams are political in one
form or another. The ACS is clearly dictating, through this exam, what it feels
to be of importance in the specific course. For example, one iteration of the
organic exam had two questions (out of 70) on green chemistry, when there was a
faculty representative on the committee who was a green chemist. The most
recent organic exam has 4 carbonyl reactions that are specific subfields and
also “named reactions” (after the chemist(s) who discovered or publicized the
use of the reactions) in part as a direct answer to a recent push in chemical
education to minimize the use of “named reactions” in the interests of
promoting a more inclusive classroom experience.
Additionally, there are issues concerning equity and
standardized exams. It has been shown, time and again in sociological and
pedagogical literature, that there are inherent equity issues around
standardized exams. There continues to be discussion and research nationwide
around what root causes exist that lead to underserved students (students of
color, students from lower economic brackets, first-generation students) and
students who identify as women to perform at lower levels on standardized
exams. Regardless of the reason, the faculty in our department believe disparities
exist. We see it in our incoming students, who benefit greatly from the
holistic application review that Utica University offers and are often high
achieving students with poor standardized test scores, and we see it on our
final standardized exams, where students who have performed exceptionally well
all semester choke on a standardized exam.
This past year, we experienced another issue with these
standardized exams: we could not vouch
for their reliability. The standardized exam results, in fact all of the
assessment data, showed little to no impact on our student education and the
2020-2021 COVID-19 experience. While that makes us go “Yay!”, we know that our
students right now have fewer skills than they would typically have at this
point in their education. We know that our students are faring better than some
cohorts at other schools; we all talk with multiple people at other
institutions and are active in ChemEd circles nationally, and we can see where
our students place. We know our students have a much smaller “COVID-lag” than
cohorts at other institutions. We are proud of that. But we know there is a
knowledge and experience gap, and our
assessment methods do not show that.
We are making the move away from the ACS standardized exams,
or we plan to use them the way we want to use them and bend them to our own
wills. There is absolutely no good reason for our program to be dictated to,
our learning goals determined and defined, by a committee of homogenous
professors protecting a status quo that we have dedicated our professional
lives to overthrowing. In our
department, the age of one-size-fits-all assessment and the lies it propagated
is over.
No comments:
Post a Comment