Assessing Teachers; What are We Really Doing?
There's an article in today's NY Times entitled, "States Try to Fix Quirks in Teacher Evaluations." I invite you to peruse the article - I'm not sure anyone would call them "quirks".
It opens with a scene probably familiar to all - a high-school LA teacher explaining what irony means. The principal was there to assess this veteran teacher's abilities. He says it was a "good lesson." BUT he had to give her a 1 (lowest score out of 1-5) because she didn't break the students into groups. He had seen her do it in the past but in her professional wisdom, she didn't feel the need to do it for this lesson. But the principal had to follow the rubric guidelines. He said:
“It’s not an accurate reflection of her as a teacher,” Mr. Ball said.
That's just one assessment of what sounds like a perfectly fine teacher. And yet, she got dinged. But it's not just the teachers:
Principals in rural Chester County, Tenn., are staying late and working weekends to complete reviews with more than 100 reference points. In Nashville, teachers are redesigning lessons to meet the myriad criteria — regardless of whether they think that is the best way to teach. And at Bearden High School in Knoxville, Tenn., physical education teachers are scrambling to incorporate math and writing into activities, since 50 percent of their evaluations will be based on standardized tests, not basketball victories.
And that last point is important because the SEA and the district just sent out an e-mail to high school teachers reminding them that:
Beginning in 2011-12 all teachers of tested subjects and grade levels will have student achievement data collected for the purpose of meeting the CBA requirements for student growth. It is essential to accurately link all students to the teacher(s) who provided their instruction so that teacher’s student growth ratings will be accurate. The designed process includes staff verification of student rosters for classes taught (Article XI Section G, p. 104).
After you return from mid-winter break, we will be asking you to review your first semester student rosters and confirm the students that received mathematics or reading instruction from you. You will be able to review these rosters and make any corrections you feel are necessary if you did not provide the instruction in one or both of these areas. Instructions for how to review and complete your roster verification will be provided when the rosters are available to review.
We targeted this data collection time to coincide with the end of the first semester. We recognize this may be an inconvenience, and want to thank you for your time and effort in completing this important task.
This is part of the CBA but it does point out the issues in assessment. Do science teachers teach math as part of their teaching? Should they submit their students' names or just math and LA teachers? It's a lot of time on the teachers' part as well as whoever has to collate all that information. (According to the note above, the teachers won't know how or who to review until they get their rosters. It seems like that doesn't give them the ability to do some of this work in advance of getting the rosters and will have to do it all at once.)
No matter how much we want to make sure who is doing good work in the classroom, there seems to be a lot of measuring to the nth degree and possible soul-searching on the part of those who do the assessing in order to be fair to a teacher.
And, is everyone good at everything? Do you do every part of your job well?
It's a brave new world and you have to wonder how many young people might reject this job because of the difficulties of understanding how they will be assessed for the work they do.
It opens with a scene probably familiar to all - a high-school LA teacher explaining what irony means. The principal was there to assess this veteran teacher's abilities. He says it was a "good lesson." BUT he had to give her a 1 (lowest score out of 1-5) because she didn't break the students into groups. He had seen her do it in the past but in her professional wisdom, she didn't feel the need to do it for this lesson. But the principal had to follow the rubric guidelines. He said:
“It’s not an accurate reflection of her as a teacher,” Mr. Ball said.
That's just one assessment of what sounds like a perfectly fine teacher. And yet, she got dinged. But it's not just the teachers:
Principals in rural Chester County, Tenn., are staying late and working weekends to complete reviews with more than 100 reference points. In Nashville, teachers are redesigning lessons to meet the myriad criteria — regardless of whether they think that is the best way to teach. And at Bearden High School in Knoxville, Tenn., physical education teachers are scrambling to incorporate math and writing into activities, since 50 percent of their evaluations will be based on standardized tests, not basketball victories.
And that last point is important because the SEA and the district just sent out an e-mail to high school teachers reminding them that:
Beginning in 2011-12 all teachers of tested subjects and grade levels will have student achievement data collected for the purpose of meeting the CBA requirements for student growth. It is essential to accurately link all students to the teacher(s) who provided their instruction so that teacher’s student growth ratings will be accurate. The designed process includes staff verification of student rosters for classes taught (Article XI Section G, p. 104).
After you return from mid-winter break, we will be asking you to review your first semester student rosters and confirm the students that received mathematics or reading instruction from you. You will be able to review these rosters and make any corrections you feel are necessary if you did not provide the instruction in one or both of these areas. Instructions for how to review and complete your roster verification will be provided when the rosters are available to review.
We targeted this data collection time to coincide with the end of the first semester. We recognize this may be an inconvenience, and want to thank you for your time and effort in completing this important task.
This is part of the CBA but it does point out the issues in assessment. Do science teachers teach math as part of their teaching? Should they submit their students' names or just math and LA teachers? It's a lot of time on the teachers' part as well as whoever has to collate all that information. (According to the note above, the teachers won't know how or who to review until they get their rosters. It seems like that doesn't give them the ability to do some of this work in advance of getting the rosters and will have to do it all at once.)
No matter how much we want to make sure who is doing good work in the classroom, there seems to be a lot of measuring to the nth degree and possible soul-searching on the part of those who do the assessing in order to be fair to a teacher.
And, is everyone good at everything? Do you do every part of your job well?
It's a brave new world and you have to wonder how many young people might reject this job because of the difficulties of understanding how they will be assessed for the work they do.
Comments
Not to mention that the CBA says in one place "student growth" and in another "test scores." Which is it? There is also lots of murky language on "accounting for special populations when looking at scores", but when it comes crunch time on evaluations I bet it will all boil down to test scores and those that work with students who are working below grade level will be damned no matter what they do or how much their students improve. Or worse your test score will be based on someone else on your "grade level team" like the teacher evaluation bill the Senate passed.
I would welcome an evaluation system that helped me be a better teacher with input from peers, students, and admin. What we have now is something that has categories like this one under classroom management that reads: "teacher shows general warmth and caring for students" (proficient) or "teacher shows genuine warmth and caring for students." (innovative) I'd love to know how the "data" on the difference between these two is collected.
As I posted elsewhere, if a student has both an LA teacher and a Reading teacher, word has it that the district will have those two teachers SHARE the credit (or "blame") for the student's growth (or lack thereof.)
How crazy is that?!
I'll post this scenario again, to illustrate the uncollaborative nature of this whole "test score" silliness. (And keep in mind, as Peony commented, that test scores are, ahem, mutable.)
Student has rising test score over a year; student had a low score last spring. Parents hired a tutor over summer, student loved tutor and responded. Parents separated in October, one parent leaving with household library (remaining parent doesn't read with student.) Student starts afterschool tutoring with volunteer in November. Student breaks up with romantic interest in December. During second semester, using MAP scores as guide, student is placed in remedial reading class in addition to regular LA. (Student is also taking AP Euro, and being challenged with complex texts.
Student also is receiving lunch-time assistance from science teacher to decode technical texts.
End of the year, student's score goes up/down - district assigns credit/blame to Reading AND LA teachers.....
It's absurd.
Anyone notice that this email to teachers was sent out a few hours before Feb. vacation? So we'd have time to read the contract and play Perry Mason the Statistician or Perry Mason the Lawyer ?? So we'd forget about it for a week? Decoding the following gibberish will consume how many hours per teacher, times how many thousands of teachers ... and will accomplish ... you've entered the twilight zone.
"When common district-wide assessments are available, the results of those assessments will be used to determine a student growth rating. These results will be calculated as set forth below.
1. Teachers of tested subjects and grade levels are those for whom two or more common state or
district assessments are available.
2. Teachers of tested subjects and grade levels will receive a rating on their student academic growth
of either low, typical, or high based on the assessments available to that teacher. Students will be
compared to their academic peers – e.g. students in the same grade who performed at a similar level
in the subject in previous years.
3. Student growth ratings will be based on a two-year rolling average.
4. Students must be enrolled 80% of the time and must be in attendance 80% of that time to have their
assessments counted in the teacher’s growth rating.
5. SPS will calculate each teacher’s rating by using a valid, reliable and transparent methodology as agreed upon by SEA and SPS.
6. To ensure that teachers of challenging student populations are evaluated fairly, aggregate student
growth results will factor in the student composition of the teacher’s classroom(s), including the proportion of English learners, students who qualify for free/reduced lunches, and students with disabilities
7. For teachers of subjects that are assessed by the state, the final rating will be contingent on the receipt of state assessment data; a written report will be issued to each teacher within 30 days of the district’s receipt of the final assessment report from the state"
OhWell
n...
I would think that we would want to set the minimum attendance rate for individual students to be about one standard deviation below that average (whatever that may be). This way, presuming a normal distribution (which is a HORRIBLE presumption, particularly in this case) about 83% of the students will be counted.
I have to wonder if Korsmo watched the same talking point vomiting Duncan I did. If she did, our differences are even greater than I thought. WSDWG
All the rhetoric about how everyone else is measured by performance in their jobs didn't include any examples reaching this degree of scrutiny, unfairness, and ridiculousness.
I'm dying for anyone, anywhere, to show me who else is rated by a similar performance metric, anywhere.
This is total bullshit, a waste of money, and the perfect dream of a bunch of deadbeat data zombies with their heads squarely up their asses.
If it isn't already readily apparent what a ridiculous burden this testing is on teachers, and how severely it interferes with student learning by stealing time and resources away from learning opportunities, then some people are simply brain dead.
Would we ever see this horse crap at Lakeside, Bill? Not on your life.
WSDWG
Of course, he's off the hook for, as he said several times, these standards were written by our own state legislators and teachers. I guess he's just the cheerleader.
n...
What a convoluted mess!!!
Can there be any better reason than this to rid SEA of Glen Bafia, J. Knapp, et al before next year's bargaining of the new contract?
The best companies/organizations don't evaluate results, they collaborate on a daily basis...
ken berry SpEd IA VA @AAA
I hope you are organizing the teachers in your school. Eric Muhs needs help. Don't hand Knapp this election.
Unintended Consequences
by Bob Dean .. Math Department Chairperson Evergreen School District.
What are we really doing?
Bob knows ... and legislators do not.
Just how scientific are tests anyway?
Concerning the teacher evaluation contract clause referencing the use of test scores, McClure teachers have a free "get out of jail" card.
Chaos rules the administration of the MAP testing process. Students are pulled nearly at random from class to be tested. Students wander in and out of the testing room almost at will.
Students are allowed to take part
of the test and then take more of the test even weeks later. Students talk to their friends about the test and share info and answers, etc.
The test is given during different times of day (squirmy students before lunch and sleepy after) and different times of the year (before or after certain topics are covered in class).
Distractions are the norm with
talking among the students and noise from other parts of the school.
The lack of scientific rigor or even minimal testing security is
breath-taking. Gathering honest or valid data under these conditions is impossible.
So kudos to Sarah Pritchett for letting all McClure teachers off the hook, rendering the onerous testing/evaluation contract clause completely invalid.
And now our wonderful teachers can now let their shoulders drop six inches and get back to teaching academics, not "testing".
McClure Watcher
I support Eric Muhs. We agree on many things, including listening to teachers before millionaires.
JKnapp may have a SpEd running mate, but know that SpEd families (of which I am one), see NO advantage to this arrangement. Political alliances this late in the game do nothing to dissuade me.