Wednesday, June 18, 2008

Two New Studies With Huge Implications

There were two new studies recently that were discussed in two articles in the NY Times.

The first article discussed a study by the College Board (which conducts both the SAT and AP tests).

"The revamped SAT, expanded three years ago to include a writing test, predicts college success no better than the old test, and not quite as well as a student’s high school grades, according to studies released Tuesday by the College Board, which owns the test."

The College Board put a positive spin on this, saying that the SAT continues to be a good predictor of first year grades in higher education. (This is somewhat true - the SAT plus high school grades - are good predictors but it also depends on what group of students you are talking about.)

“The 3-hour, 45-minutes test is almost as good a predictor as four years of high school grades, and a better predictor for minority students.”

But critics had this to say:

"But critics of the new test say that if that is the best it can do, the extra time, expense and stress on students are not worth it.

“The new SAT was supposed to be significantly better and fairer than the old one, but it is neither,” said Robert Schaeffer, the public education director at FairTest, a group that is critical of much standardized testing. “It underpredicts college success for females and those whose best language is not English, and over all, it does not predict college success as well as high school grades, so why do we need the SAT, old or new?”

The other study was more troubling. The title of the article? "Report Sees Cost in Some Academic Gains.

"The study, to be released on Wednesday, compared trends in scores on federal tests for the bottom 10 percent of students nationwide with those for the top 10 percent and said those at the bottom moved up faster than those at the top.

In tests of fourth-grade reading from 2000 to 2007, for instance, the scores of the lowest-achieving students increased by 16 points on a 280-point scale, compared with a gain of three points for top-achieving students, according to the study, by the Thomas B. Fordham Institute, a research organization in Washington.

The period of big gains for low achievers and minimal ones for high achievers coincides with the federal education law known as No Child Left Behind, which took effect in 2002. The study said that while it was impossible to know whether the law caused those scoring patterns, such a result would hardly be surprising, since the law made it a goal to reduce the gap separating low-scoring, poor and minority students from higher-scoring white students."

(One more thing to add to the "to do" for Secretary of Education Margaret Spellings, who according to yet another NY Times article, is trying desperately to get NCLB revamped before the end of Bush's term (good luck with that). She seems a little like Terry Bergeson as both women have hung their professional lives on one thing - assessments.)

The report is to be released in two parts; one outlining how lower-achievers have moved ahead more than high achievers and the other about how teachers seem to recognize who it is they are supposed to be helping the most.

"The report included results of a survey of a nationally representative sample of 900 teachers. Seven in 10 teachers said their schools were more likely to focus on struggling students than average or advanced students when tracking achievement data and trying to raise test scores. And about three-quarters of the teachers surveyed said they agreed with this statement: “Too often, the brightest students are bored and under-challenged in school — we’re not giving them a sufficient chance to thrive.”

Amy Wilkins, a vice president at Education Trust, which lobbies for policies to help close the achievement gap, said the gains by low achievers should be applauded. “My concern is that this report makes it seem like we have to choose between seeking equity and excellence,” she said. “We need to strive for both.”

I haven't read the report yet. And Ms. Wilkins is right to applaud forward progress (but, of course, if it had been high achievers moving forward 16 points and lower achievers moving ahead 3 points, there would have been howls) but the point is we need to work for all students across the board.

"Susan Traiman, director of education policy at the Business Roundtable, a group that represents business executives, said the challenge was to improve the ability of schools to educate students across a range of levels.

“We’re producing progress at the bottom, and we need to maintain that,” Ms. Traiman said, “but we need to ratchet up the performance of students at every achievement level if we’re going to be competitive.”


zb said...

Presumably this would be addressed in a real study, but one would expect, mathematically for gains at the top of the scores to be less than gains at the bottom of the scores, especially given that such scores are likely to see saturation effects. That is, some portion of kids in the top 10% are scoring the equivalent of 800s (or 2400s) on the SATs, so they have no where to move up at all.

The numbers themselves would not cause me to interpret the result being at all the fault of NCLB, unless a different pattern was seen before.

My 1st grader apparently has the reading level of a 10th grader. If you look for improvement of her scores over the next 11 years, and presuming that reading levels actually max out at 12th grade, she's not going to move up very much.

(I haven't read the study, so my criticism might not apply, and they might address it. But, the numbers you summarize don't mean much)

Melissa Westbrook said...

I might expect that, too but given that the teachers in the accompanying study (75% of them) say that they devote more of their time to lower achieving students because they were trying to achieve higher test scores.

That tells me that they feel the pressure more on the bottom end rather than trying to help every child because of testing and that they are willing to admit (likely with frustration) that they are not helping all students.

ZB, maybe you think there's no room for improvement in your child's learning (or maybe you just mean in reading) but from a broader perspective, every child needs to encouraged and pushed.

hschinske said...

I have no problem with the top level of scores on a grade-level test being stagnant. There are many other ways to test whether students at the top are being challenged appropriately. I'd much rather they didn't *try* to use the grade-level tests to do what they were not designed to do.

Sure, there are problems with high-achieving students getting neglected under NCLB, but the test score stagnation is the wrong data to be looking at, because it's going to occur anyway.

Helen Schinske

zb said...

I don't know if there are problems with high achieving students getting neglected, because I'm not convinced that a teacher's time is the right metric.

I do think that a high-achieving child deserves to be encouraged and pushed, too, but sometimes that can be done with very little time. It depends on the child, and what they want to do.

But, the real point I was trying to make is Helen's -- test scores on "in level" tests mean little when you're looking at the educational needs/success/progress of high achieving children.

zb said...

and, no, I don't believe my 7 year old, though mighty bright, has reached her highest level of cognitive function (or at least I hope not). :-)

The point was not whether there's room for improvement, but whether standard testing can measure it