How will the District measure student growth for the purposes of teacher evaluation? That's not entirely clear, but we do have some hints:
From the letter to teachers dated 8/3/10
Student learning and growth will be based in both teacher-determined and District-determined data and measures, and will account for the fact that not all children are the same. The District measure will be based on the overall growth of a teacher's students relative to students of similar demographics who have performed like them in previous assessments, and will be calculated as a two-year rolling average on at least two student assessments.I'm not sure what this means.
A. "the overall growth of a teacher's students" I'm not sure how they will measure "overall" growth for a teacher's students. For elementary students I suppose they mean in various disciplines (reading, writing, math, etc.). Do they mean the same for secondary students? Will the District measure of student growth for math teachers, for example, include the students' growth in other disciplines?
B. "relative to students of similar demographics" So the District's expectations for student growth will vary based on race and gender. What other demographic grouping will they make? Free and Reduced Price Lunch Eligible? English Language Learners? How is this going to work and how will it be expressed? Will the District expect greater student growth from some demographic groups than others? Why don't we have the same high expectations for all of our students? Don't we believe that all students can learn and can reach the same high levels of achievement?
C. "who have performed like them in previous assessments" This appears to suggest that student growth targets will be set by historic outcomes for various demographic groups. What form will that take? What will this mean in concrete terms? Will the teacher get a report that says: "Student A is a fourth grade Latino boy who is not FRE or an ELL. Based on previous assessments for students in this demographic group, we expect this child to advance 0.8 years in math, 0.9 years in reading, and 1.1 years in writing." And the teacher's effectiveness will be measured relative to that expectation of student growth? Do we even have historic outcomes for students? The MAP assessments have only been done for one year. Does one year of data - the initial year - constitute a reliable data set? With just one year of data, can the District speak about "previous assessments"?
D. "will be calculated as a two-year rolling average" What is calculated on a two-year rolling average - the target or the individual student growth? It can't be the student growth, otherwise the teacher's evaluation would be based in large part on the previous year, before the student was in the teacher's class. It must be the District's targets for growth. But the District doesn't have two years of MAP data to average yet, so what will they use the first year?
E. "on at least two student assessments" Oh! So they won't rely on a single test of student growth. They will use the MAP and... well, what else? They don't have anything else that claims to be able to measure student growth. The MSP or HSPE (the tests formerly known as the WASL) don't measure student growth at all; they are criterion-referenced tests. So what else will the District use? Maybe they could use Classroom Based Assessments (CBAs) if they ever fulfill their dream of standardizing them - oops! I mean aligning them. Will this require teachers to do the District-written and approved CBAs? Or do they mean that student growth will be measured using the math MAP and the reading MAP? Or do they mean that student growth will be measured using last year's MAP and this year's MAP?
In the FAQ sheet on SERVE, the District re-states many of the words, but doesn't clarify the meaning very much:
Individual student growth: Individual student growth will include two types of measurements, each based on the extent to which a teacher’s students meet or exceed scores on standards‐based assessments that are typical of their academic peers (students who have performed like them in previous assessments).This statement is very similar to the one before, but rather than providing clarification, the subtle, interesting differences muddy the water a bit.
• District‐required: A teacher’s score will be based on a two‐year rolling average of the overall growth of his or her students relative to their academic peers in at least two common assessments.
1) "the extent to which a teacher’s students meet or exceed scores". This suggests target scores instead of target growth. Hmmm.
2) "standards‐based assessments" The MAP is not a Standards-based assessment. The only Standards-based assessment is the former WASL, now known as the MSP and the HSPE. Hmmm. These assessments, however, because they are criterion-referenced tests designed to assess the effectiveness of schools and districts, are utterly inappropriate for measuring year-over-year academic progress for individual students.
3) "typical of their academic peers (students who have performed like them in previous assessments)" Now we have a different definition of academic peers - instead of demographics, they are using "students who have performed like them". So now student growth will be relative to students who achieved similar scores last year. The inherent problem here is that, by definition, half of the scores will be below the median. How would this look in real life? Last year Student B scored a 330 on the third grade reading MSP. This year Student B scored a 360 on the fourth grade reading MSP. The average score on the fourth grade MSP for students who scored 330 on it in the third grade was 350, so this is regarded as evidence of the teacher's effectiveness because Student B's score met or exceeded the score typical of Student B's academic peers. Did I read that correctly?
If I understand these statements correctly, and if they mean what I think they mean, then I'm not impressed with this as a measure of teacher effectiveness. It relies on a perfectly dreadful misuse of assessment data. It relies on unfounded beliefs in correlations (if not causation) between student assessment data and teacher effectiveness.
Worse, I don't think the District folks know what they mean by any of this. I don't think they know what assessments they will use or how they will use them. I don't think they have decided on the peer groups or how to determine relative growth for individual students. I, for one, would like to see all of this spelled out and I would like to see a backtest before I agreed to any of it.
They can and should perform a backtest. In fact, I don't see how they propose this thing without having conducted a backtest. The backtest would be an example of how these measures would have appeared for a few teachers last year. Choose a few classrooms from schools across the District with a diversity of students and programs, and see how the District-determined measure of student growth would have worked and been scored for each of them. What would be the District-determined measure of student growth for a third grade Spectrum teacher at Lafayette, a kindergarten teacher at Gatzert, a math teacher at McClure, a Language Arts teacher at the NOVA Project, a high school teacher at Middle College, a Special Education teacher at Lowell, and a fifth grade teacher at View Ridge? Let's see some concrete examples so we have some idea of what is proposed and so the District has some experience doing the calculations.