What's Worse Than Assessments?

German is remarkably efficient at capturing complex emotions in a single word.  The weight gain associated with a breakup or other emotional turmoil is called kummerspeck—literally, grief bacon.  Americans have appropriated the German word schadenfreude –taking pleasure in the misfortunes of others—I assume because it so perfectly captures the driving force behind much of our popular culture: reality television, tabloid journalism, the 24-hour news cycle.

I recently learned another German word that I feel needs to be embraced, if not by the culture as a whole then certainly by those of us in education.  Schlimmbesserung is an improvement that makes things worse.  I entered the teaching profession in 1995 and, since that time, have lost count of the number of times I've witnessed or been the victim of schlimmbesserung. 

Last month, a participant in one of my beginning teacher seminars asked my opinion of a new policy in his high school.  The faculty had been tasked with administering common formative assessments, or CFAs, that would provide baseline data against which they could measure student progress.  Concern arose that the students were simply marking random answers on their CFAs, which the leadership team interpreted as a failure to take the process seriously. 

(Il)logic Model

Before I reveal how they "solved" the problem, let’s first examine their logic.  I present the following syllogism:

1.  Students marked random (incorrect) answers because they didn’t take the process seriously.
2.   If students took the process seriously, more of their answers would have been correct.  
3. Different answers would provide better data.

Now, this explanation applies if, and only if, the students both were and believed themselves more capable than their responses suggested, thus making their incorrect responses acts of pure defiance.

Research ethics, however, demand that we examine all the possible explanations of the data.  Is it possible that the students’ answers only seemed random because they didn’t know what the right answers were?  Alternatively, is it possible they marked random answers because they knew they lacked the necessary knowledge or skills to do otherwise?  In Why Don’t Students Like School?, cognitive psychologist Daniel T. Willingham explains that people enjoy thinking but only “if we judge that the mental work will pay off with the pleasurable feeling we get when we solve a problem.”  Absent the belief that effort will yield the dopamine rush of success, most people will give up.  This is why I no longer even attempt sudoku puzzles.

If the students didn’t try because they either knew or suspected that their effort would not result in a payoff, there is no reason to believe that greater effort would have resulted in more correct answers.  More importantly, the administrative team ignored the very solid data their students provided, that, at best, they lacked confidence or motivation and, at worst, were incapable of correctly answering the questions.  “What forest?  I don’t see any forest!  There’s too damn many trees in the way!”

To incentivize the students, the principal mandated that their scores on the CFA be counted as 10% of their semester grade.  It is bad enough that those outside the profession dictate policy that is counterproductive to our students' needs.  I find it inexcusable that an instructional leader would demonstrate such disregard or ignorance of the function of formative assessment.  These are tools to collect data  regarding a student’s progress toward a learning goal which is then used to adjust instruction.  Formative assessments should never penalize students for not knowing information they have not yet been taught.  Nor are they tools for social engineering.

Even if we allow, for argument’s sake, that the original premise was correct, and the students randomly marked their tests because they didn’t take them seriously.  Is it reasonable to then assume that these same students will feel differently if you threaten them?  

Conclusion

I served my time in a public-school classroom and am therefore not unfamiliar with the many ways teenagers try to game the system.  They can be real asshats.  That impulse to test the boundaries and see what they can get away with is kind of what makes them teenagers.  What separates us—the adults—from them is our ability to resist the urge, however strong, to respond to their impulsive, short-sighted behavior with impulsive, short-sighted behaviors of our own.  

Educators do not take an oath of office, but maybe they should and, like doctors, pledge to, first, do no harm. 

What if I told you there were no U.S. schools?

Among the sticks used to beat teachers in this country is the perception that U.S. students are less educated than their international peers.  Those who make this claim usually cite the findings of large international assessments, such as the TIMSS or PISA studies, that show the mean score of American students well below those of other industrialized nations.  Here’s the problem, though.

From an instructional standpoint, there are no U.S. schools.  There are no American students.

 “Whaaat?!” I hear you cry.  “Of course we have U.S. schools.”

No, we don’t.  We have a Department of Education—a fairly recent addition to the Presidential cabinet.  With the smallest staff but third largest budget of any cabinet-level department, it is empowered to lead the dialogue about education and disperse funds for various programs and grants.  

Now, you might be thinking, “Doesn’t receipt of those funds entail some sort of compliance measure or attainment of some standard?”

And the answer is, “Mmm...meh.”

Let’s take a recent and well-publicized example, the Race tothe Top program, part of the American Recovery and Reinvestment Act of 2009.  Among the stipulations attached to the receipt of these funds were the adoption of common standards and the use of high quality common assessments.

Had these stipulations been enacted, we might be on the road to having U.S. schools.  Instead, they became plot devices in a political drama that pit the liberal/tyrannical Obama administration against states with conservative/freedom-loving legislatures.  Those states voted essentially to renege on the terms of the deal. 

Most of the decisions about the enactment of teaching and learning are made at the state or district level.  That is why, in some states, science teachers must present the argument for intelligent design along with evolution or why other states use textbooks that refer to African slaves as imported workers.

The difference between the states becomes unmistakable when their international achievement scores are disaggregated and each state’s results are presented separately.

The graphic seen here is taken from U.S States in a GlobalContext: Results from the 2011 NAEP-TIMMS Linking Study.  It reveals that in eighth grade mathematics, Massachusetts students, on average, are outscored only by students from South Korea, Singapore, Taiwan, Hong Kong, and Japan.

Also noteworthy is the success of many states in comparison to Finland.  Until this three-ring circus that constitutes our Presidential election eclipsed everything else, I couldn’t pull up my Facebook feed without reading about the idyll that is is Finnish education.  According to the NAEP-TIMSS data, though, more than half of the states in this country, including several of the most populous, outscore Finnish students in math at the eighth grade level.

Now, I know that test scores do not provide a complete picture of a school system.  However, test scores are the preferred metric of those who criticize “American” schools.  By and large, these are the same people who shriek, “Government overreach!!” in response to any effort to ensure that students in one state receive the same quality of education as students in another. 

This year will mark my 21^st as an educator, and I am officially tired of defending my profession.  Alexis de Toqueville asserted that, in a democracy, people get the government they deserve.  Perhaps in a confederation of quasi-independent states, they get the education systems they deserve as well.