The Academic Assessment Committee
recently reviewed the gift-giving process practiced at the North Pole. What
follows is the committee’s feedback and suggestions regarding these practices.
Santa has articulated an outcome
("Children should be nice"), but the wording is too ambiguous, and,
therefore, hard to measure. Further, he has not specified an appropriate
target. Should 100% of the children be "nice" 100% of the time? 90%?
75%? Without a clearly defined target, Santa risks being arbitrary and
inconsistent in his assessment of children's behaviors.
The methods Santa uses to assess
children's behavior and distinguish between "naughty" and
"nice" are not apparent. How often is he able to observe each child
first-hand? Are there others who are involved in this assessment?
Santa's elves or Mrs. Claus, for example? Given how high stakes this assessment
is, there should be multiple individuals engaged in observing and evaluating
children's behavior, and there should be 90% inter-rater reliability. Further,
each individual child's behavior should be observed on numerous occasions
throughout the assessment cycle. This may not be a sustainable plan, however,
particularly given the recent cuts made to the workshop staff as a result of
the pandemic and the shortages caused by the supply chain problem.
It is also not clear what instrument
is used to document children’s' behaviors. Does Santa use a rubric that articulates
clear criteria regarding the kinds of behaviors he regards as "nice"
versus "naughty?" Do these criteria take into account cultural
differences? In other words, are Santa's assessment practices
equitable?
The results of Santa's assessments
have never been published or analyzed. What percent of the world's
population under the age of 10 gets what they asked for? What percent
receives coal in their stockings? Are there specific trends that Santa has
observed over a period of time--say the last 100 years? Has the
percentage of naughty children increased? Decreased?
Since the purpose of assessment is
to inform improvement, how are Santa's findings shared with and used by
parents? Or do his "naughty" and "nice" lists remain in a
drawer in his office at the North Pole and referenced only during the Christmas
season as a prop? How might parents (and perhaps even teachers) use them
to help develop children’s character?
The committee recommends that Santa reflect
more deeply on his assessment processes and consult with other characters such
as the Easter Bunny and the Tooth Fairy to get ideas for how he might design an
assessment plan that is fair, useful, and sustainable.
