Assessment as Inquiry into Student Learning

When the assessment movement started in the mid-1980s, its advocates envisioned a paradigm shift in higher education, a move away from the “professor-as-disseminator-of knowledge” model and towards one that promoted active, collaborative learning.  For this approach to succeed, however, faculty needed to articulate clearly what they wanted students to learn and consider carefully how well they were learning it.    

If one approaches assessment as something to satisfy institutional requirements or accreditation standards, then assessment is little more than tedious bean-counting that yields meaningless quantitative information.  On the other hand, if one regards assessment as its original proponents intended it to be—deliberate and thoughtful inquiry into student learning—it can result in important discoveries that improve pedagogy and curriculum.     

Consider the assessment process as analogous to any other process of discovery, whether creative, scientific, or problem-solving.   It starts with defining the problem or asking the question:  How do we know students are learning this material or developing this ability?  The next “step” is to investigate or observe.  In the creative process, this might involve research or brainstorming; in the problem-solving process, like the assessment process, it means gathering data and any other evidence that may possibly answer the question.  Once ideas are generated or data have been collected, patterns begin to emerge.  These patterns provide insights and direction; they shape meaning.  When we are writing, this might be when we discover a lead or focus.  In assessment, these patterns provide insight into where students are performing well, and where they are less successful at meeting the learning goals.  This doesn’t necessarily happen after one assessment, any more than it happens in writing after reading one research article.  Patterns emerge only after sustained and systematic data collection.

Reflection is critical to any process of discovery, and assessment is no exception.  Reflecting on and interpreting what results might mean are important; this cannot be emphasized enough.  Likewise, formulating solutions to address specific findings, particularly those that are disappointing or below target, is an integral part of the discovery process.  Positive discoveries are made from assessment as well, just as they are through the scientific and creative processes, and assessment should be a way to celebrate these successes. 

As with any process of discovery, our inquiry into student learning will have its limitations.  But perfection cannot prevent us from doing something well.  Our commitment to an educational mission is a commitment to thoughtful analysis of student learning, to asking the right questions and exploring the possible implications inherent in what we discover. 

Capturing A Rich Narrative: Experiential Learning Opportunities

If assessment provides a way of telling our story, then tracking experiential learning opportunities is probably one of the most exciting parts of the narrative.

By “experiential learning,” I am not referring to a good or even great experience, like taking students to an art museum or engaging them in a community service activity for one afternoon.  I am talking about those hands-on experiences that occur over a period of time and enhance deeper learning.  As many of the departmental assessment reports document, these high impact experiences are integral to a Utica College education.

In a number of academic departments, these types of experiences result in student presentations at regional or national conferences.

• Last October, 3 students attended the Seaway Section of the Mathematical Association of America (MAA) meeting at the University of Toronto Mississauga.   This spring, 1 student will present at the MAA Seaway Section meeting at St. John Fisher. 

• From 2017 through 2018, 5 chemistry students presented their research at the American Chemical Society’s national conferences, and one presented at the CSTEP Statewide Conference. 

• 15 students have been included as co-authors on presentations made at regional and national psychology conferences from 2017-2019. Two students have also been included as co-authors with a faculty member in a prestigious professional journal publication.

• In the geoscience program, students engage in field trips during lab periods and on weekends. They also participate in internships, independent research, and may opt for a 4 to 6 week field camp experience to study the geologic features of a particular region.  In 2017, 2 undergraduates presented posters at a professional conference, and 1 student’s research was published in Northeastern Geographer. 

Experiential learning isn’t realized solely in conducting research and giving presentations, however.  Students are writing for the Tangerine.  They are performing on stage in musicals and dramatic productions.  They are studying abroad.  They are completing internships.  And sometimes experiential learning happens right in the classroom or during residencies, as in the case of the Financial Crime Management program.  In this program, graduate students get hands-on experiences using computing software and financial analysis tools and applying them to real-world criminal cases in economic crime.

Experiential learning exposes students to new opportunities and often takes them outside their comfort zones.  In MGT/PRL 345, students spend spring break in New York City, where their instructor has arranged for them to visit with UC alumni and other top communications professionals at agencies such as G & S Business Communications, the Wall Street Journal, Glamour, NBC News, the New York Power Authority, and the 9/11 Memorial and Museum.  Student reflections indicate that this experience is a transformative one, especially for those who come from small, rural towns where opportunities are limited and who have never visited a large city.  One student wrote, “In college, it’s hard to figure out where you firmly belong or it’s difficult to see yourself in five years.  But when you visit an [organization] and you feel like you could belong there, it’s an empowering feeling.”

Now if these aren’t impressive outcomes, I don’t know what are.  

Formative Assessment

Formative assessment refers to the approaches instructors use in their classrooms to determine what students are understanding or not understanding.  It is and has always been integral to effective teaching. Christopher R. Gareis from William and Mary’s School of Education notes, “What we now call the ‘Socratic method’ essentially amounts to using questions to assess understanding, to guide learning, and ultimately, to foster critical thinking” (Gareis).  Socrates’ persistent questions represent one method of formative assessment.

Methods of formative assessment are diverse.  They include having students summarize what they learned on a 3 x 5 index card before leaving class; asking them to build or create something that shows they are able to apply what they learned; requesting them to provide feedback or respond to a question using a clicker or Twitter voting; urging students to complete a self-assessment of their work, using the same rubric or matrix that the instructor does.  The important aspect of formative assessment is that it is a way to guide instruction and provide feedback to students.  Whether it is graded or not is the prerogative of the individual instructor. 

Utica College’s physics department offers an excellent example of how formative assessment engages students, stimulates curiosity, and promotes a sense of community.  Since 2011, the department has offered a 1-credit seminar where students and faculty read a book relevant to the discipline and engage in online discussions and face-to-face conversations, the latter facilitated by students.  Faculty participate less as “experts” and more as members of a learning community, exploring themes and new ideas in collaboration with the enrolled undergraduates.  These ongoing discussions and the attendant questions and responses represent formative assessment at its finest.  In a recently published article, “A Multilevel Seminar for Physics Majors:  A Good Deal for Everyone,” the physics faculty describe how this approach to student learning has enhanced student engagement in the learning experience and resulted in student growth. 

In the MBA program, new students are required to complete a one-page essay where they analyze their reasons for pursuing an MBA degree.  Each essay is scored using the AAC & U Value Rubric for Critical Thinking.  The scored rubric is intended to provide students with feedback on their critical thinking and writing skills. It also introduces them to the expectations of graduate-level work and familiarizes them with the criteria that will be used to assess their work.  This formative assessment serves another important purpose as well:  it is used to refer students to services and resources that might be used to support them in their graduate coursework.

A faculty member in Wellness and Adventure Education brings experiential learning into his traditional classroom by using “real-world” projects and simulations.  After students engage in a group activity, they reflect on their performance, providing each other with feedback and insights.  Students reflect further in writing on the experience and what they learned both from the experience and their peers’ feedback. 

These kinds of formative assessments provide texture to the assessment narrative. So let’s hear from you.  What are you doing in your classroom or in your program to guide instruction and give students feedback on their learning?    

Works Cited

Gareis, Christopher R. "The Forgotten Art of Formative Assessment." February/March 2006. William and Mary School of Education. 23 January 2019.

L.S. Dake, J. Ribaudo, and L.H. Day. "A Multiplevel Seminar for Physics Majors: A Good Deal for Everyone." The Physics Teacher. December 2018: 630-632.

UC Assessment: Thinking Beyond the Rubric

A departmental mission statement should identify the program's distinguishing characteristics. Utica College's Foreign Language department's mission does precisely that.  The mission statement highlights a unique feature of the department's curricula, namely that learning a language other than English prepares students for their professional lives, regardless of career choice, by developing intercultural competence and empathy.

This mission is realized in a capstone project that requires students to integrate their language learning with either a practical experience or a scholarly endeavor.  Recent examples include projects in pedagogical approaches to teaching a language (education), risk management in Mexican agriculture (business/risk management), bullfighting in Spain today (history), and literary and cinematographic representations of El Camino de Santiago.  These capstone projects provide a rich learning opportunity for students and are an ideal opportunity for faculty to assess not only how well students have achieved the program-level learning goals, but also how effectively the department is fulfilling its mission.

Assessment strategies in the Chemistry Department show how assessment should not be limited solely to students' performance in individual courses.  The department measures student success by using multiple methods, including standardized examinations, course-embedded projects, student research, graduates' employment status, and placement in graduate or professional schools.  While assessment findings have been used to inform changes in the curriculum, so, too, has the faculty used "best practices" to inform their work.  A recent example documented in the department's 5-Year Program Review has to do with chemical safety.  Faculty member Alyssa Thomas learned of resources for chemical safety through her work with the governance branch of the American Chemical Society.  She worked with her colleagues in the department to develop a coherent process of safety instruction.   This included articulating clear goals for each laboratory course and implementing methods to assess these goals at various stages of student development.  The faculty noted "dramatic improvement" in students' awareness of safety measures in the chemistry lab.

Using Blogs and Simulations to Explore Global Perspectives

By Dan Tagliarina

Increasingly as educators we are asked to address, or find ourselves using language of, “global perspectives.” While exactly what we mean by global perspectives is not always clear, what is clear is that there is a drive—one I argue that is appropriate for a liberal arts education—to understand different viewpoints and to recognize the legitimacy of a variety of positions on different topics where evidence can be reasonably interpreted in different ways. This post is not about these views directly, but rather draws on two types of assignments I use in my classes to get students to think through other perspectives. These assignments are blogs and in-class simulations

Part of coming to terms with other perspectives is also understanding forms of communication. Different forms of communication can have different purposes, along with their own rules and norms. In other words, not all writing is created equal or meant to do the same thing. A note scrawled on a piece of paper and left on your roommate’s desk is not the same as a text sent to a friend or a research paper submitted for an upper-level class. This should be obvious. What is often less clear about this is that for each of these forms of writing (and so many more) to be effective the writer needs to think about who the audience is, and what the point of the communication is. Is it inquisitive (do you want to grab dinner tonight after class)? Is it informative (my car won’t start; I’m not going to class)? Is it scholarly (For the foregoing reasons, I argue, it is highly illogical to conclude that the moon is actually a malevolent force causing humans to periodically misbehave)?

Considering the intended audience and intended message requires students to think through what to say based on how it is likely to be received. This is where blogging assignments come in. In many of my upper-level classes I have students maintain their own blogs, posting weekly for most of the semester on a topic of their choosing related to the course topic. The idea is to have students engage in learning outside of the classroom, and share this learning with a public audience. For this activity students must practice a form of writing—public online discourse—that inherently requires thinking about how the audience will perceive the information. The audience is not fully known, so students must think about who the audience might be, and what these potential perspectives are (while still completing an assignment for a specific class). Also, as a less formal form of writing, students are given the opportunity to communicate in their own style, while trying to be informative. It is my hope, in part, that this assignment has students thinking through the perspectives of their potential audience, as well as their own perspective, as they try to engage in a form of public discourse.

In turning to in-class simulations the connection to global perspectives is more direct. By having students engage in simulations of any kind—in my classes these often take the form of moot courts—we are asking the students to inhabit a specific role and perspective to perform some assigned task. Basically, we are asking students to be someone else. This requires students to think through other perspectives, and in the case of moot courts, often make arguments with which they might not agree. In researching and performing roles that are not just “student” the students can expand their own horizons, breakdown the typical confines of books and classrooms, and start to think about how classroom learning becomes applicable outside of campus. In this sense, simulations become a simulacrum for an aspect of the non-collegiate world. Combining any simulation with post-simulation reflection papers than furthers the chance to have students engage in metacognition and consider how the simulations help them to think more globally, even if on a small scale, as they engage in other experiences and consider other viewpoints.

Blogging and simulations can push students towards global perspective as these activities, in their own ways, ask students to think through the implications of interactions and to change their perspective for viewing these interactions. Global perspectives, in this context, could be beyond the U.S., it could be beyond the local, at the very least it is beyond the confines of the typical college learning environment. Both blogs and simulations are about thinking through various other perspectives: those of readers, including different audiences; portraying other roles engaging in specific behavior.

Then the challenge becomes assessing whether students are doing any of this. For me, I am looking for if the students are able to think through various other positions, and understand these other positions. These assignments are not about forcing views on anyone, or requiring students to accept new ideas. These assignments are aimed at having students reflect on what they, and others, think and believe, and in so doing come to a better understanding of their own positions and of the positions of others. Recognition of various perspectives is at the heart of democratic norms required for a thriving civil society. That is what I see as the goal of pursuing “global perspectives,” and one that is both achievable and measurable. How “global” these global perspectives are will vary from class to class, and professor to professor, but the goal should be to broader our students horizons and push them to understand multiple viewpoints.

Using Assessment Results: A Shout Out to Academic Departments

Gathering evidence of student learning or program effectiveness is only a means to an end. Assessment results are meaningful when they are useful and relevant. What follows are some excellent examples of how departments and faculty have used assessment to strengthen program offerings or make inquiries into student success.

The Biology Department examined two years’ worth of results from graduating seniors’ performance on the Major Field Achievement Test (MFAT) and discovered that students’ knowledge of ecology (especially at the levels of community, ecosystem, and biome) was an area of weakness.  The faculty then examined the curriculum map for the biology major and noted that students are only exposed to material in ecology in the general biology course, unless they elect to take an ecology course.  This discovery had obvious implications for curricular changes and/or modifications to the major requirements, including adding ecology as a required course for the major.

The departmental faculty also discussed the learning outcomes for the capstone courses offered in the major. Since students may choose the course they wish to complete, the faculty determined that the learning goals should be comparable for each. Specifically, one capstone course was redesigned as a writing intensive course to ensure its consistency with the other capstone experience.

The Physics Department recently submitted a proposal to the Curriculum Committee for a 1-credit optional course designed to give students practice applying concrete problem-solving strategies to a wide range of physics problems. This curriculum change was made when the faculty observed that a number of students struggled with the math required in PHY 151 and were unable to complete the homework required to become proficient with the subject matter.

The History Department made a significant curriculum change by changing a 1-credit gateway course to a 3-credit one.  This modification was informed by student performance in the capstone course. Not only will this change allow for a “broader, deeper, and more comprehensive foundation for the methods-and-research spine of the program,” it will also allow the faculty to implement a more sustainable assessment plan that measures growth in the major.  

MBA faculty see assessment as integral to the scholarship of teaching and learning (SoTL). Stephanie Nesbitt, Matt Marmet, and Tracy Balduzzi will have the results of one of their assessments published in the spring edition of The Business Research Consortium Academic Journal of Education. This article, titled “The Impact of Behavioral Engagement on Outcomes in Graduate Business Blended Learning Environments,” is part of a larger study on student engagement/student success. They will also be presenting their work in both poster form and a 60-minute presentation at the Teaching Professor Conference in Atlanta, GA this June.

David Fontaine implemented the Athlete Viewpoint survey to measure coaches’ performance, student-athletes’ satisfaction, and student learning outcomes.  He used the findings to transform the athletic program at UC.  Michael Cross, Penn State University Assistant Director of Athletics and co-founder of Athlete Viewpoint, commended Fontaine’s work in a blog published at ultimatesportsinsider.com.  Cross writes, “Having real data and analytics should be incorporated into all areas of [an] athletic program to enhance performance, mentor, lead, and support decision making.”  He credits Dave Fontaine with doing just that, and he urges other athletic directors to follow this leadership example. 

How I Talk About Citizenship in the Classroom

By Austen Givens

I can think of few contemporary topics less fashionable to discuss in the classroom than good citizenship.

But I am also hard pressed to identify any period of American history since World War II in which discussions of good citizenship were as important, or more vital, than now.

I teach courses on homeland security and cybersecurity at Utica College. Most of my students aspire to positions of public trust. Alumni from our suite of academic programs—consisting of criminal intelligence analysis, criminal justice, cybersecurity, and financial crime investigation—enter law enforcement agencies, the military, intelligence agencies, banks, and insurance firms, for the most part. Virtually all of these positions demand that our alumni exhibit the qualities of good citizens.

So, what are these qualities? And how do I teach them?

Five traits—or virtues—spring to mind: honesty, integrity, justice, love of public service, and respect for the U.S. Constitution.

I use assignments, activities, and lead by example to promote these qualities of good citizenship.

Honesty

The free exchange of ideas is vital to healthy democracies. So, I encourage a free exchange of ideas in my classes. That free exchange extends to views that some could find offensive or vile. Reasonable adults may disagree forcefully. Yet they can do so in a civil, professional manner. If a student wants to air a controversial viewpoint, I let her do it. And if other students wish to attack that controversial idea, I gently—but firmly—encourage them to remain focused on the idea itself, not the person who aired it, and to demolish the idea with evidence and facts.

Integrity

I try to model integrity for my students. Over the years I have made my share of mistakes in the classroom. I’ve written unclear quiz questions, for instance. And it is almost always a student, not me, who identifies these poorly written quiz questions. I try to convert these mistakes into a learning opportunity. I publically thank the student for pointing out the issue with these questions, even if the student has been gracious enough to bring it to my attention privately. And I bias corrective action in favor of students’ interests. Those ambiguous quiz questions? I usually give the entire class credit for them, regardless of their actual responses. If I expect my students to act with integrity, then I am convinced that I must show them what integrity looks like in action.

  

Justice

To me, justice means doing what is fair and proportionate consistently. In a graduate course that I teach, CYB-667 – Critical Incident Command, Response, and All-Hazards, I have students participate in a peer exchange activity to reinforce this. Students produce a written report, then swap these reports with their classmates and score them using a rubric that I provide. This is designed to do at least two things: first, it helps students to understand the value of peer editing; and second, this activity permits students to assess work in an objective way. Evaluating a situation, a person, or a report with reference to objective standards is an exercise in justice, besides also being a highly competent way to work. Cultivating justice in this way reinforces a good habit that can be used over a lifetime.

Love of Public Service

I am persuaded that public servants must love public service to be truly effective. In practice, this means that public servants must learn to act in ways that are ultimately consistent with the public interest. That requires critical thinking skills.

I’ve taught multiple classes at Utica College about terrorism, such as CRJ-305 – Terrorism and CRJ-307 – Homeland Security and Counterterrorism. In written assignments in these classes I try to re-direct students’ attention to what may be unspoken or downplayed—the constitutional rights of terrorism suspects, for example, or the costs associated with prosecutors seeking the death penalty in terrorism cases rather than long prison sentences.

I do this because in the world of counterterrorism acting in the public interest often means restraining government action. This notion runs contrary to the let’s-blow-em’-to-smithereens political rhetoric about terrorism that we often hear in popular media.

But more importantly, it is about learning to act first with the public interest in mind.

Respect for the U.S. Constitution

I try to emphasize to students that basic principles like the freedom of speech (1^st Amendment), the right to bear arms (2^nd Amendment), and protections against unreasonable searches and seizures (4^th Amendment) are woven into the fabric of American life.

For example, much has been made of hate speech and “fake news” on the Internet, and these topics are often fodder for written discussions in my online classes.

Calls to strengthen or relax Constitutional provisions to deal with these problems inevitably return to questions about the Constitution itself. The Constitution may set forth impossible-to-achieve ideals. If we are to move forward as a society, however, we must at least respect the Constitution. For whatever flaws it contains, it also provides well-designed guardrails for life in a republic.

***

These are some of the tools and techniques that I use to teach good citizenship at Utica College. Do you do anything along these lines? Do you have any new activities or ideas that you could recommend to make good citizenship come alive in the classroom? Let me know!