Sunday, April 22, 2012

Pineapples Don't Have Sleeves

If ever there was a reason to pull a stop on standardized testing mania, it's this story.

The Hare and the Pineapple.

Yes, apparently in NYC's 8th grade standardized test there was a reading section, based on a story by Daniel Pinkwater, the children's author, about a hare and a pineapple having a race, based on The Tortoise and the Hare.  (The story was altered from Mr. Pinkwater's original according to Mr. Pinkwater.)

So the Pineapple challenges the Hare to a race and as all the other animals are standing around, the Crow says the Pineapple has something up his sleeve because the Pineapple can't move.  (It is also pointed out that pineapples don't have sleeves.)

So when Hare arrives and takes off, they are confounded but then the Hare finishes the race and everyone cheers and eats the Pineapple.  The End.

What?

Naturally, if you are an 8th grader who can actually READ, you may have a problem figuring out the moral of the story.

The Education Commissioner says the media didn't print the whole thing and "it makes more sense in the full context of the passage" but admits the questions are "ambiguous."   Here's the whole thing

Mr. Pinkwater told the New York Daily News that it's the "world's dumbest test question."

Apparently this question has been used in several different states and confused kids in all of them. 

Now if the use of this story is to challenge kids to think differently about motivation or find nuances in stories, there are better stories. 

Here's a 1931 8th grade test for comparison from The Answer Sheet blog at the Washington Post.

From Valerie Strauss of the Washington Post:

The problem, of course, isn’t one test question that people think was badly drawn, or the strong likelihood that other questions on these exams make little sense or actually assess only a small band-width of skills, concepts and knowledge that we want students to know.

The problem is that the results of standardized tests are being used in New York and other states to assess not only students but teachers, principals and schools through complicated formulas that purport to show how much “value” a teacher adds to a student’s achievement. Researchers say that “value-added” assessment models can’t do what supporters say they do and are unreliable accountability.

The stakes of these tests are getting higher as educator evaluation systems are being put in place that are based largely on how well a student does on these exams.

15 comments:

seattle citizen said...

Ever since 2001, I have been mightily suspicious of high-stakes test questions. Here is the disgusting, insulting, enraging, question that caused me to distrust the construction of all of these tests since (and that they are now being used to evaluate not only students, but also educators via further twisted data "attribution" and manipulation, just adds to their craptitude):
NY Times 4/29/01: Test Answer Calls to Mind A Scandal, by Sam Howe Verhovek
"For Washington students taking a state standardized test...the question seemed innocuous enough, if not particularly exciting.
Using the geographical information provided about four imaginary towns along the route of a school bus driver, figure out the alignment of the towns from west to east.
The correct answer was: (C) Mayri, then Clay, then Lee, then Turno. Mayri. Clay. Lee. Turno.
Wait. Say that out loud. Does that sound like Mary Kay Letourneau?"

Jack Whelan said...

It might be theoretically possible to design a test that fairly assesses student and teacher performance, but in the current divisive political/cultural climate it's a very, very remote possibility. I don't think in this instance it's a question of letting the imperfect to be an enemy of the good, but of the potentially very harmful delivering at best some very marginal benefits.

The costs, both in blood and treasure, don't justify this testing obsession. But that so much rides on these tests attests not to what is best for student learning, but what meets the agendas of institutional actors who care mostly about their careers.

BTW: For what it's worth, my ranking of the super candidates, given what we know so far about them, is Enoch, Banda, and a distant third--Husk.

In an earlier thread I said I'd like someone with local roots at the end of his or her career who will bring experience, maturity, and who will not look at this job as a stepping stone to somewhere else. Enoch comes closest to that description, and other things I've heard about him make me think he will be the best fit insofar as he doesn't seem to be aligned with any national political agendas and will bring energetic, imaginative leadership balanced with seasoned common sense.

That's my raw first impression, but I'd like more info about his stance toward NCLB and RttT, and corporate reform in general. I think it's good that he has some private-sector experience and understands first hand how that world works, but I want more assurances that he understands that the constituencies he serves are, first, students and their families (and their elected board); second, teachers and staff; and a distant third--the downtown political and business establishments.

mirmac1 said...

There are two interesting articles on School Finance 101 blog about the complete failure of VAM, and even worse failure of SGP.

For those of you like me, who wouldn't know an SGP if it slapped me across the face, those are STUDENT GROWTH PERCENTILES. I recall during a board presentation Eric Anderson was talking about using the Colorado Growth Model SGPs for teacher evaluation because, why not, they all making it up as they go along anyway.

Anonymous said...

after skimming an article about Charles Taylor, and learning what 'short sleeves' and 'long sleeves' meant to his butcher henchmen - maybe a good multiple guess question could be:

What is 1 of the surest ways of destroying civil society, being led by

A. The Brutal, or
B. The Stupid, or
C. The Brutally Stupid, or
D. Arrogant elites, or
E. huh?

UhHuh?

Anonymous said...

after skimming an article about Charles Taylor, and learning what 'short sleeves' and 'long sleeves' meant to his butcher henchmen - maybe a good multiple guess question could be:

What is 1 of the surest ways of destroying civil society, being led by

A. The Brutal, or
B. The Stupid, or
C. The Brutally Stupid, or
D. Arrogant elites, or
E. huh?

UhHuh?

Lisa said...

Wow. Just wow. I hope everyone clicked through and read the asinine questions about the Pineapple and Hare story. It's a silly story, but it is not nonsensical and it would be possible to ask non-ambiguous questions about it. Question 1 is OK, for example, although maybe simplistic for 8th grade.

My child had a practice MAP test packet over break. Here is one question:

Paneet checked the thermometer at 8:00am. The temperature fell 26 degrees by 6:00pm. (then there is a picture of a thermometer that reads about 48 degrees)What is the temperature at 6:00pm? Choices: 22F, 26F, 32F, 48F

There is no notation of what time of day the pictured thermometer is supposed to represent. We figured the answer had to be "22" but with no more info than what was given it really isn't possible to choose with confidence.

Anonymous said...

As a parent of a kindergartner, I really do not understand the value of the MAP test. Her fall scores changed a lot (upward) when scores were recalibrated. Her winter reading score went down even though I can tell she had learned a lot between fall and winter. It just made no sense to me and it was a waste of her time and the schools resources. I would rather she spent more classroom time with her amazing teacher actually learning something than sitting in front of a computer taking a test with confusing questions.

Also, how is it that some schools can send home test preparation packets and other schools do not? Yet another way these results will be potentially skewed as some schools choose not to test in the fall, prep kids with take home worksheets, and spend time doing pre tests on the computer before taking the real test. You can't tell me that does not affect scores. It is such a waste of time and resources.

Frustrated parent.

J said...

I think anybody who's spent any time with kindergarteners would agree that their test scores can't be considered remotely reliable. I shudder to think that important decisions are being based on the scores of 5-year-olds.

It only underscores the feeling that the reform industry really doesn't know that much about education.

Anonymous said...

MAP practice packets? And time spent practicing the tests on the computer? Wow. Neither of my kids' elementaries do that. Thank goodness.

Catherine

Sahila said...

@J - you make the mistake of believing/assuming that the reform industry is even remotely interested in real education- its not... you yourself call it an "industry"... its not about the kids - its about how to make the most profit...

someone posited the theory yesterday that testing companies like Pearson who design standardized tests design them with two goals in mind: one is to make millions and the other is to make sure our kids fail them!! So they can reap the rewards of making more tests to fail thereby holding kids back in school so they stay in school longer to take more tests that will dumb them down even further...

Anonymous said...

Is it possible they were practicing for the MSP, not MAP? My child's school is doing the computer based MSP this year (no more test booklets), and there was some instruction on computer use.

a mom

Anonymous said...

Wow. My kid's practice test for the ISEE (a standardized test for private school admissions) included passages from Great Expectations, and non-fiction descriptions of ancient Egypt. And this is for 5th graders. I can't believe such a simplistic reading passage, and such inane and nonsensical questions, for an 8th grader.

Stunned

Jan said...

Stunned: I haven't looked at the ISEE lately, but I totally agree with you. Unbelievably simplistic reading passage. And even worse -- totally inane and nonsensical questions. Looking at old WASL questions, I also noted LOTS of instances of the "thermometer problem" noted by Lisa above -- where the questions are larded with ambiguities, or make unstated assumptions that kids have to guess about, etc. Without knowing whether the kids correctly "guessed" what assumptions to make -- and how the authors meant the ambiguities to be resolved, there is no way to fairly grade the questions. They are so riddled with vagueness that they render the answers useless even for aggregated purposes, much less for purposes of evaluating a specific student -- or even worse, the teaching abilities and success of that student's teachers.

Sahila said...

‎#Pineapple Revolution Is ON!!!


Palm Beach Co Schl Bd FLA adds support to national movement vs over emphasis on high-stakes testing Sun-Sentinel FCAT Frenzy

Jan said...

Jack -- I liked all of your points (and agree with your preliminary ranking of candidates). My position on testing is that the test that would be fair is so theoretical that it should be taken off the table. I would elaborate, but the whole thing is so complex that it gets too long. Kids and teachers both need to be evaluated. But unless you evaluate the accuracy of the test with respect to each kid -- it is unfair to both the teacher and the child. And we don't seem to have a system (or the resources) to do that.