What works: Not for Everyone

A recently released review of beginning reading programs from the What Works Clearinghouse found that few commercial reading programs could claim evidence that they are effective in raising student achievement (Kathleen Manzo, Education Week, “Reading Curricula Don’t Make Cut for Federal Review, August 15, 2007). This isn’t particularly surprising to those who have studied commercial reading programs, but it raises a question that is seldom asked: what does it mean to claim that a reading intervention “works?”
To begin with, no reading intervention has been found to be effective with all children, all of the time. So a reading strategy that “works” does not work for everyone. From a statistical point of view, strategies work only for a mythical average student. The reliance on means for determining statistical significance obscures the fact that a strategy that was found to work did not work for everyone and may even have been detrimental for some.
The claim that a reading intervention works must be further qualified with the phrase, “compared to what?” Typically, reading research compares one intervention to one or two other interventions. In some cases, the intervention may actually be compared to nothing (i.e., the intervention is better than no intervention). In any case, a strategy that works only works better than the interventions to which it was compared and, even then, because of the reliance on the mythical average student, a strategy that didn’t work could still be effective for some children.
Finally, the assertion that a reading intervention works begs the question, “works at what?” Some researchers will be satisfied that an intervention was effective if it improved students’ performance sounding out nonsense words. Others will only be satisfied if the intervention improved students’ reading comprehension and, even then, reading researchers have different views on the meaning of reading comprehension.
So to say that a reading intervention works really means that the intervention was effective for some children compared to one or two other interventions on measures the reading researcher(s) – but likely not all reading researchers – believed were related to reading.
From this perspective, the ultimate arbiter of “what works?” is the teacher who determines the efficacy of various reading interventions with individual children in her/his classroom.