Testing - What Does it Reveal?
Psychology Today put forth this article by researcher Susan Engel, "What Test Scores Don't Tell Us: The Naked Emperor." Ms Engel reviewed over 200(!) studies of K-12 standardized tests.
What I have discovered is startling- most tests used to evaluate students, teachers, and school districts predict almost nothing except similar scores on subsequent tests. I have found virtually no research demonstrating a relationship between those tests and measures of thinking on the one hand, or life outcomes on the other. To grasp what we do and do not (yet) know about standardized tests, it’s worth considering a few essential puzzles: why we find individual differences in test scores (why one child does better or worse than others), what makes a child’s test scores go up, and what such improvement could possibly indicate.
Most researchers agree that several non-school factors have a big impact on children’s performance on academic tests.
What? TFA says those can be overcome. Ditto Bill Gates. How could these researchers be wrong?
Children who don’t hear much language at home are at an academic disadvantage, which is manifested, among other places, in their test performance. Children whose parents read a lot do better than children whose parents don’t read.
To that end, this fascinating article from the NY Times about that very idea - how many words in his/her vocabulary does the average kindergarten bring to the first day of school?
As the education theorist E. D. Hirsch recently wrote in a review of Paul Tough’s new book, “How Children Succeed,” there is strong evidence that increasing the general knowledge and vocabulary of a child before age 6 is the single highest correlate with later success. Schools have an enormously hard time pushing through the deficiencies with which many children arrive.
However,
But most of us believe that intelligence and family background do not seal a child’s fate. We believe that children can learn something in school which gives them knowledge and skills above and beyond what they can get on their own. Furthermore, the current faith in testing suggests that we believe that test scores are a good measure of whether children are learning something valuable at school.
When scores go up, assuming no cheating is involved, people tend to think it means that a specific teacher or educational practice has helped children to know more and think better than they otherwise would have.
But do we have evidence of this? I haven’t seen any. To show that improved test scores actually indicate a more knowledgeable and skilled child, we need at least three kinds of evidence.
First, we need evidence that when a child scores better than she has in the past, her knowledge or skills extend beyond the specific items on the test.
Second, it would be good to know that when children’s test scores improve, their academic performance in non-test settings also improves. In other words, we’d need evidence that the teacher whose students regularly get better scores than predicted by their earlier scores are also become better thinkers and learners more generally.
Third, even in the absence of these two kinds of research, it would be good to know that improving a child’s test score actually improved their life outcome.
A good read.
What I have discovered is startling- most tests used to evaluate students, teachers, and school districts predict almost nothing except similar scores on subsequent tests. I have found virtually no research demonstrating a relationship between those tests and measures of thinking on the one hand, or life outcomes on the other. To grasp what we do and do not (yet) know about standardized tests, it’s worth considering a few essential puzzles: why we find individual differences in test scores (why one child does better or worse than others), what makes a child’s test scores go up, and what such improvement could possibly indicate.
Most researchers agree that several non-school factors have a big impact on children’s performance on academic tests.
What? TFA says those can be overcome. Ditto Bill Gates. How could these researchers be wrong?
Children who don’t hear much language at home are at an academic disadvantage, which is manifested, among other places, in their test performance. Children whose parents read a lot do better than children whose parents don’t read.
To that end, this fascinating article from the NY Times about that very idea - how many words in his/her vocabulary does the average kindergarten bring to the first day of school?
As the education theorist E. D. Hirsch recently wrote in a review of Paul Tough’s new book, “How Children Succeed,” there is strong evidence that increasing the general knowledge and vocabulary of a child before age 6 is the single highest correlate with later success. Schools have an enormously hard time pushing through the deficiencies with which many children arrive.
However,
But most of us believe that intelligence and family background do not seal a child’s fate. We believe that children can learn something in school which gives them knowledge and skills above and beyond what they can get on their own. Furthermore, the current faith in testing suggests that we believe that test scores are a good measure of whether children are learning something valuable at school.
When scores go up, assuming no cheating is involved, people tend to think it means that a specific teacher or educational practice has helped children to know more and think better than they otherwise would have.
But do we have evidence of this? I haven’t seen any. To show that improved test scores actually indicate a more knowledgeable and skilled child, we need at least three kinds of evidence.
First, we need evidence that when a child scores better than she has in the past, her knowledge or skills extend beyond the specific items on the test.
Second, it would be good to know that when children’s test scores improve, their academic performance in non-test settings also improves. In other words, we’d need evidence that the teacher whose students regularly get better scores than predicted by their earlier scores are also become better thinkers and learners more generally.
Third, even in the absence of these two kinds of research, it would be good to know that improving a child’s test score actually improved their life outcome.
A good read.
Comments
It's an old saw in analytical sciences that a "model" is "a set of assumptions" -- and that it's key to identify assumptions, especially the ones that we don't realize we're making. Deep in the fooferaw about MAP testing, over and over I see some variant on this: "It's not that testing isn't useful, it's that this test isn't useful."
But is "testing" useful? Believe me, I understand the siren song of data, mounds, reams, freakin' PILES of data, but it would seem that, for most "tests", one might as well use sunspot activity to drive one's education policies.
It's a grim fact that teaching and learning simply don't boil down to a handful of replicable metrics. I believe that in a very real sense, good teaching is like the classic definition of pornography: we know it when we see it -- and the corollary is, we know when we're not seeing it, too. But muscling the whole process into a nice three-dimensional box is an exercise in futility: it just doesn't WORK that way. Making policy based on the assumption that it DOES work like that is a horrifying waste of money, time, and intellectual capital.
From the Washington Post, letter from high school teacher to college professors about the impact of high stakes testing on student quality. Worth reading.
K5STEM mom