The Trouble with DIBELS

I’ve been grading my students’ research papers the last few days and I’ve been surprised how many students made some reference to the use of DIBELS in their schools. What particularly surprised me was the fact that some of these students taught in affluent, high-achieving schools. I had assumed that DIBELS was mainly used in underachieving, Reading First schools. It seems that I was wrong.

In the forward to Ken Goodman’s book, The Truth About DIBELS: What It Is - What It Does, David Pearson warns that “DIBELS is the worst thing to happen to the teaching of reading since the development of flashcards” (p. v). Pearson takes this strong stance because, in his opinion, “DIBELS shapes instruction in ways that are bad for students” and “bad for teachers.” Pearson believes that DIBELS is bad for students because it shapes instruction in ways that do not promote students’ development as readers and bad for teaches because it requires that teachers shape instruction “based on criteria that are not consistent with our best knowledge about the nature of reading development” (p. v).

Perhaps the most widely used – and troubling – DIBELS measure is Oral Reading Fluency which produces a measure of students’ reading speed and accuracy (number of words read correctly per minute). What’s troubling is the behavioral theory of reading that underpins this measure. DIBELS is informed by a developmental model of reading that assumes learning to read is a matter of learning to sound out letters and words to a level of “automaticity” and, once children read with sufficient fluency (speed and accuracy), they will be able to comprehend what they’ve read.

The theory of reading underpinning DIBELS predicts that fluent readers should be good comprehenders and, conversely, students who are not fluent (do not read with sufficient speed and accuracy) should be poor comprehenders. In their book, Rereading Fluency: Process, Practice, and Policy, Bess Altwerger, Nancy Jordan, and Nancy Rankie Shelton report a study they conducted that, among other things, examines the relationship between reading fluency and reading comprehension. What they found was that some of the best comprehenders read slowly. Similarly, some of the most fluent readers (those who read the most words correctly in one minute) were among the poorest comprehenders.

Altwerger, Jordan, and Shelton’s research challenges the basic theoretical assumptions underlying DIBELS. Their research also reinforces the argument that “scientifically-based research” isn’t just about sound methods. It is also about sound theory and the theory of reading on which DIBELS is based is fundamentally flawed.

What works: Not for Everyone

A recently released review of beginning reading programs from the What Works Clearinghouse found that few commercial reading programs could claim evidence that they are effective in raising student achievement (Kathleen Manzo, Education Week, “Reading Curricula Don’t Make Cut for Federal Review, August 15, 2007). This isn’t particularly surprising to those who have studied commercial reading programs, but it raises a question that is seldom asked: what does it mean to claim that a reading intervention “works?”
To begin with, no reading intervention has been found to be effective with all children, all of the time. So a reading strategy that “works” does not work for everyone. From a statistical point of view, strategies work only for a mythical average student. The reliance on means for determining statistical significance obscures the fact that a strategy that was found to work did not work for everyone and may even have been detrimental for some.
The claim that a reading intervention works must be further qualified with the phrase, “compared to what?” Typically, reading research compares one intervention to one or two other interventions. In some cases, the intervention may actually be compared to nothing (i.e., the intervention is better than no intervention). In any case, a strategy that works only works better than the interventions to which it was compared and, even then, because of the reliance on the mythical average student, a strategy that didn’t work could still be effective for some children.
Finally, the assertion that a reading intervention works begs the question, “works at what?” Some researchers will be satisfied that an intervention was effective if it improved students’ performance sounding out nonsense words. Others will only be satisfied if the intervention improved students’ reading comprehension and, even then, reading researchers have different views on the meaning of reading comprehension.
So to say that a reading intervention works really means that the intervention was effective for some children compared to one or two other interventions on measures the reading researcher(s) – but likely not all reading researchers – believed were related to reading.
From this perspective, the ultimate arbiter of “what works?” is the teacher who determines the efficacy of various reading interventions with individual children in her/his classroom.