Saturday, August 21, 2010

A Plethora of Reading

So most of you may have heard that the LA Times is doing a huge multi-part story about teacher evaluation. One of the biggest parts is a listing of every single public school teacher and their classroom test scores (and the teachers are called out by name).

From the article:

Though the government spends billions of dollars every year on education, relatively little of the money has gone to figuring out which teachers are effective and why.
Seeking to shed light on the problem, The Times obtained seven years of math and English test scores from the Los Angeles Unified School District and used the information to estimate the effectiveness of L.A. teachers — something the district could do but has not.

The Times used a statistical approach known as value-added analysis, which rates teachers based on their students' progress on standardized tests from year to year. Each student's performance is compared with his or her own in past years, which largely controls for outside influences often blamed for academic failure: poverty, prior learning and other factors.

Interestingly, the LA Times apparently had access to more than 50 elementary school classrooms. (Yes, I know it's public school but man, you can get pushback as a parent to sit in on a class so I'm amazed they got into so many.) And guess what, these journalists, who may or may not have ever attended a public school or have kids, made these observations:

On visits to the classrooms of more than 50 elementary school teachers in Los Angeles, Times reporters found that the most effective instructors differed widely in style and personality. Perhaps not surprisingly, they shared a tendency to be strict, maintain high standards and encourage critical thinking.

But the surest sign of a teacher's effectiveness was the engagement of his or her students — something that often was obvious from the expressions on their faces.

Mr. Kotter! The surest sign of a teacher's effectiveness was the engagement of students. And the reporters back this up with...nothing. How do they know this? I have known some pretty entertaining teachers who were not that good at teaching.

(I'm sorry but anyone reading this blog has just as much ability to assess a classroom as reporters from the LA Times so what they say on this point means little to me.)

What the union has to say:

In an interview last week, A.J. Duffy, president of United Teachers Los Angeles, was adamant that value-added should not be used to evaluate teachers, citing concerns about its reliance on test scores and its tendency to encourage "teaching to the test." But Duffy said the data could provide useful feedback.

"I'm not opposed to standardized tests as one means to helping teachers look at what's happening in their classrooms," he said.

The next story is a piece in Seattle Crosscuts by Councilman Tim Burgess. Look, I've talked to Burgess, I think he's a smart guy, a well-meaning guy (and likely Mayor of Seattle someday) but he doesn't really understand education (or hasn't tried to at least get a well-rounded understanding). First, he kind of takes issue on the op-ed piece that he and Councilman Richard Conlin had written for the Times recently. He says that people were personally critical of him and this is true but only because most of the commenters (myself included) did not believe they wrote the piece. (Frankly, I think they read a couple of white papers, talked to some ed reformers and called it a day, passing off the writing to aides.)

Then he goes on to say - finally - that:

The reforms needed in public education involve the entire system, not just teachers. Everyone involved in public education — administrators, principals, school board members, and teachers — is responsible for the system we have today that routinely fails a third to one-half of our children. There is enough failure for everyone to share.

But there is no one word of HOW that accountability will come. Thanks, Tim. And, then he goes on to talk just about teachers. He then says the line that Charlie hates a lot:

Teachers are the single most important factor in a child's education, so discussion about reform often centers on how they are evaluated, rewarded, and recognized.

I agree with Charlie; parents are the single most important factor in a child's education. Decades of education research state this and I believe it is far more likely that researchers have found parents rather than teachers the most important factor.

Hilariously, next to the Burgess piece is a link to an article from the Wall Street Journal entitled, "Needs Improvement: Where Teacher Report Cards Fall Short."

From the article:

One perplexing finding: A large proportion of teachers who rate highly one year fall to the bottom of the charts the next year. For example, in a group of elementary-school math teachers who ranked in the top 20% in five Florida counties early last decade, more than three in five didn't stay in the top quintile the following year, according to a study published last year in the journal Education Finance and Policy.

"Because education tends to have this moral-crusade element…we tend to rush to use things before they are refined or really fully baked," says Frederick Hess, director of education policy studies at the American Enterprise Institute, a conservative think tank.

But even skeptics of test-score-based evaluations acknowledge that a uniform, data-based approach for ranking teachers could be superior to subjective methods—such as principals' observations—that still predominate in schools. "Damn near anything is going to be an improvement on the status quo," says Daniel Willingham, a cognitive psychologist at the University of Virginia.

Sounds like there's some confusion and uncertainty here. But okay, let's try something different but can we make sure first that we do no harm?

Also:

These measures don't simply ding teachers for their students' low scores, because not all incoming classes start the year equally. Instead, teachers are evaluated based on how much students' scores improve by the end of their year.

But good teachers aren't easy to identify this way. For one thing,students aren't always assigned to teachers randomly. A teacher who gets more than his share of students who learn slowly because of his knack for helping them might be penalized at the end of the year.

39 comments:

Megan Mc said...

A couple of questions that came to mind when reading this post:

1. Value Added - what if the 4th grade curriculum/materials are better aligned to the standards than the 3rd grade curriculum/materials? If this is the case then 3rd grade teachers will always look bad because the kids will score better in 4th grade based on materials/curriculum decisions that may be out of the teachers control.

2. If teachers are the single greatest factor in a student's education then shouldn't said teachers have more control to do what they know is right? If the assertion is true, then politicians should stay out of curriculum and materials adoption decisions.

3. The next question would naturally be; Shouldn't teachers have the MOST say about how their school/district is governed? But most teachers I know are uninterested in the politics of education and this I think hurts them because their expert voice is missing from policy decisions that directly affect them.

Sahila said...

Burgess, I think, is listed as a member of the Our Schools Coalition...

Sahila said...

Members of the Our Schools Coalition include:
the Alliance for Education, Central Area Motivation Program, CEER (Coalition for Equal Education Rights), CCER (Community Center for Education Results), Cheryl Chow, former Seattle School Board President, Council President Richard Conlin, Councilmember Tim Burgess, Councilmember Bruce Harrell, Councilmember Mike O’Brien, East African Community Services, Greater Seattle Chamber of Commerce, Horn of Africa Services, Kevin C. Washington, Chair, Tabor 100 Education Committee, King County Hispanic Chamber of Commerce, League of Education Voters, Mona H. Bailey, retired Seattle Public Schools District Administrator, The New School Foundation, Partnership for Learning, Powerful Schools, Rainier Scholars, Seattle Breakfast Group, Somali Community Services of Seattle, Stand for Children, Technology Access Foundation, Technology Alliance, Urban Enterprise Center, Urban Impact, Urban League of Metropolitan Seattle, Washington Policy Center, WTIA and Youth Ambassadors....

http://www.partnership4learning.org/node/2747

dan dempsey said...

As we look at all this talk about evaluating teachers and what is most important.......

consider this:
"Give them proper content in an accessible format and they will learn it. Don’t and they won’t.

There are instructional practices and materials that have shown huge gains but these are seldom used.

Now we are focused on SERVE .. what a farce.

An unusable test as the basis for the implementation of a plan that has yet to produce positive results anywhere.

Note the way teachers bounced around in the graph from Florida in the WSJ.

What needs to happen is to have principals do their jobs .. and put that $4 million annually into the classroom.

Note EVERYTHING proposed by MGJ always requires a LARGER CENTRAL ADMINISTRATION.

How's that working out?

gavroche said...

So it has come to this. Modern "ed reform" has deformed into McCarthyism.

This is nothing less than a witch-hunt against teachers.

How would the LA Times' reporters and editors like a bunch of teachers sitting in their cubicles with them, watching and assessing their work?

What qualifies a reporter to judge a teacher?

Test scores are the most narrow and shallow measure of a child's true educational growth. A good teacher does much more than create good test-takers.

At this point, the so-called "ed reformers" have lost any remaining credibility. They are a downright creepy bunch of hypocrites who may as well be carrying pitchforks.

gavroche said...

http://eyeoned.org/content/accountable-to-whom-d-c-schools-chancellor-fires-teachers-based-on-value-added-measures_54/#more-54

Interesting read from Prof. Aaron Pallas about (Broad Foundation Board Member) D.C. Chancellor Michelle Rhee's recent mass firing of teachers based on test scores and the imprecision of "value-added" measurements:

"(...) many states, seeking to improve their chances at winning a share of the $4.3 billion Race to the Top honey-pot, fell over themselves to enact legislation that would link teacher evaluations to student test scores, and would expand testing to the early elementary grades and high school grades so that more teachers could be evaluated on the basis of their students’ test performance.

This in spite of a substantial body of scientific evidence and expert opinion urging caution in using student test scores for making high-stakes decisions about teachers.

Among the key critiques: the tests were neither designed nor validated for use in evaluating teachers [THIS IS TRUE FOR THE MAP TEST HERE IN SEATTLE - IT IS NOT DESIGNED FOR EVALUATING TEACHERS, YET SUPT. GOODLOE-JOHNSON WANTS TO USE IT FOR THAT PURPOSE, AS STATED IN HER "SERVE" PROPOSAL TO THE UNION]; a teacher’s location in the value-added distribution can bounce around from one year to the next, or from subject to subject; students may be assigned to teachers’ classrooms in ways that invalidate the assumptions of the value-added models; a teacher’s value-added score may be sensitive to which student and school factors are included or omitted in the model; and the value-added models produce estimates that are riddled with error or uncertainty, yet are often treated as though they’re precise measurements.

kprugman said...

"what if the 4th grade curriculum/materials are better aligned to the standards than the 3rd grade curriculum/materials? If this is the case then 3rd grade teachers will always look bad because the kids will score better in 4th grade based on materials/curriculum decisions that may be out of the teachers control."

What if all the teachers were using "Everyday Math" and subtracting from learning?

Who has yet to question such an obvious gaff in policy? The entire field has been poisoned. There is nothing left that can be trusted.

Sahila said...

More reading... this time from the Washington Post:

Three books about education reform
By Diane Ravitch
Sunday, August 22, 2010

Now that the Obama administration has invited the states to compete for $5 billion in stimulus funds, the winners will not be those that come up with the best reform ideas, but those that agree to do what the administration wants: create privately managed charter schools, evaluate teachers by their students' test scores, and close low-performing schools. Since so much power and money are arrayed on one side of the issue, it is useful to consider some dissenting views. These three books have the power to change the national discussion of what now passes for "school reform."

1 Linda Darling-Hammond's "The Flat World and Education: How America's Commitment to Equity Will Determine Our Future" (Teachers College, $21.95) contains a valuable lode of practical and research-based advice about how to improve our schools. Darling-Hammond does something that the Obama administration has failed to do: She reviews what the top-performing school systems around the world do to get great results. Their highest priorities, she shows, are building a strong, experienced staff and making sure that every school has access to a rich, well-balanced curriculum in the arts and sciences. Finland, the highest-performing nation, has not relied on testing and accountability to achieve its current status.

2 Barbara Torre Veltri's "Learning on Other People's Kids: Becoming a Teach for America Teacher" (Information Age; $29.99, paper). If American education has a sacred cow, it is Teach for America, which recently won $50 million from the U.S. Department of Education. The organization recruits bright college graduates to work for two years in the nation's poorest schools. Veltri has taught many of these recruits in her job at the University of Arizona, and she interviewed hundreds for this book. While she admires the young people who join the program, she raises important questions about the value of placing unprepared teachers in classes with the nation's neediest children.

3 If I were assigning reading to staff members at the U.S. Department of Education, I would ask them to study Richard Rothstein's "Grading Education: Getting Accountability Right" (Teachers College and Economic Policy Institute; $19.95, paper). Rothstein and his colleagues explain in plain language why current accountability policies, which focus only on basic skills, are making education worse, not better, by narrowing the curriculum. With apt examples, they also show how the pursuit of numbers distorts more important goals and how schools may get higher test scores without supplying better education.

Diane Ravitch is a former assistant secretary of education. Her latest book is "The Death and Life of the Great American School System: How Testing and Choice Are Undermining Education."

dan dempsey said...

Here you go I am providing a bolded version of the Conclusion of the Value Added Management Approach to evaluating teachers.

This conclusion comes from the "Education Finance and Policy Journal" at MIT Press.

You will find that Performance Management MGJ SERVE style is another poorly researched proposal from TEAM MGJ. If you do the research at MIT Press you will find how little work anyone at TEAM MGJ did or at least how little intelligence was used in pushing this idea forward.

There is an entire issue of the Journal devoted to the topic .. fortunately the Document I am referencing was free. HERE.

Here is my take on this issue:

Once again Seattle Admin’s Performance Management proposals appear to be another Giant Seattle Experiment put forth as if the experiment is based on reliable proven relevant data, when in fact no such evidence exists. The similarity to the other Giant Experiments like the Cleveland three year math experiment, the District’s k-12 math program, the 6-12 Denny/Sealth school, the coming NTN project based Cleveland STEM and so many other poorly research proposals can hardly be missed.

dan dempsey said...

The above conclusion is taken from:
Volume 4, Issue 4 - Fall 2009 - Special Issue:
Key Issues in Value-Added Modeling


Would Accountability Based on Teacher Value Added Be Smart Policy? An Examination of the Statistical Properties and Policy Alternatives

by Douglas N. Harris
in Education Finance and Policy
Fall 2009, Vol. 4, No. 4: 319–350.

=============
WOW!!! Once again TEAM MGJ missed the alternatives .. likely because they ignore research. Just like a majority of the school board ignores research.

Research ...We send it and the Board ignores it.

kprugman said...

Arne's reform - Either the teacher's go, or the students go. Bad teachers and low-scoring students are wasteful and shouldn't be holding the rest of the schools back. If public education drops half the students that will free up more money for our strongest supporters, the charter schools.

dan dempsey said...

Here is the Abstract from the above Journal Article:

Abstract
Annual student testing may make it possible to measure the contributions to student achievement made by individual teachers. But would these “teacher value added” measures help to improve student achievement? I consider the statistical validity, purposes, and costs of teacher value-added policies. Many of the key assumptions of teacher value added are rejected by empirical evidence. However, the assumption violations may not be severe, and value-added measures still seem to contain useful information. I also compare teacher value-added accountability with three main policy alternatives: teacher credentials, school value-added accountability, and formative uses of test data. I argue that using teacher value-added measures is likely to increase student achievement more efficiently than a teacher credentials-only strategy but may not be the most cost-effective policy overall. Resolving this issue will require a new research and policy agenda that goes beyond analysis of assumptions and statistical properties and focuses on the effects of actual policy alternatives.
=================

Looks like another MGJ plan that is not cost effective ... No surprise there.

dan dempsey said...

Sahila posted:

"Darling-Hammond does something that the Obama administration has failed to do: She reviews what the top-performing school systems around the world do to get great results. Their highest priorities, she shows, are building a strong, experienced staff and making sure that every school has access to a rich, well-balanced curriculum in the arts and sciences. Finland, the highest-performing nation, has not relied on testing and accountability to achieve its current status.

Originally Linda Darling-Hammond was considered to be among the strongest candidates for Obama's Secretary of Education. So she didn't make it because of....
(1) Poor jump shot
(2) The Billionaire Boy's Club figured Arne Duncan would be easier to get to buy into total crap.

reader said...

"Parents are the single biggest contributor to educational success"... is like saying: "clean water is the biggest medical advance in the 20th century". It's a bit of a duh-so-what statement, albeit true. When you seek out a good doctor, do you say... "By golly, I've got clean water so the doctor doesn't matter." ??? Of course not. I'm sure Charlie and Melissa don't throw up their hands and say... I'm a really great parent so the program and teacher my kid gets really doesn't matter. And if it matters to them, (obviously it does), why shouldn't it matter to everyone? And if matters to everyone, why not reward performance? Nobody is saying it's easy... just that the constant refrain asserting parent supremacy is idiotic, and irrelevant.

Melissa Westbrook said...

But Reader, how do you really feel?

The point is that no one IS saying anything about parents and what happens at home. No one (or else they give it lip service).

I absolutely don't follow your train of thought, though. You go from a program to a teacher to what? Honestly, I don't see your point.

Dorothy Neville said...

Melissa, I think I understand Reader's point and I have to agree.

There are some valid talking points regarding whether or not to reward teachers for performance. There's a valid discussion to be had about whether that's a good idea or not or HOW it could be done in a valid way (if at all).

But folks chiming in about how parents are more important than teachers for a kid's success is pretty much a non-sequitur in that discussion and it gets kinda tired.

Why should "they" be saying anything about what happens at home? We are talking about education, not social work. What happens at home is beyond the control of schools, so why not keep the discussion within that realm?

The only place I can see that it is valid is if we are trying to come up with a valid "value added" performance measure and one would have to factor in home life.

BUT, if we go too far in that direction, then you are back where we started before NCLB, when the culture of low expectations was widely accepted. NCLB, for all it's faults, tried to obliterate that sort of thinking, giving up on kids based on their home situation.

kmk33 said...

It was so sad to read that this data was never made available to teachers prior to this LA Times article, who then decided to take it all public.

Personally, I would love to have access to the trend of my students' performance on standardized tests. It would be one of many pieces of information that I could use to help me identify strengths and weaknesses in my practice. And, from the experience of the Board certified teacher mentioned in the article, a piece that there isn't another way to identify.

That said, it was troubling that the reporters equated quality teaching with only these test scores, which is ridiculous.

I would be interested to know why parents sought out these specific teachers, what is it that they value that they can measure through less formal channels, and why didn't the reporters address these attributes as part of the teachers' over-all effectiveness? Would knowing the results of test scores change these parents' minds?

reader said...
This comment has been removed by the author.
reader said...

Well, the same rhetoric (parents vs teachers) applies to programs too doesn't it? I mean, if "parents are numero uno"... then neither the teacher nor the program matter much, do they? Why bother with either... if they really don't matter? Except, people have their pet programs they wish to support.

The studies don't say anything about the influences of the "impact of parents"... because that's obviously NOT something that schools can control. Duh. Sure, we could broaden the discussion. But then it would be SaveOurPublicPlanet... not SaveOurPublicSchools.

Schools are supposed to serve everybody, remember? No matter who your parents are, remember? To simply throw up your hands and say... "Oh well, the parents are big idiots. So sad, too bad. Schools did their best for me, teacher must have been good." defeats the point of public education.

Melissa Westbrook said...

"Well, the same rhetoric (parents vs teachers) applies to programs too doesn't it? I mean, if "parents are numero uno"... then neither the teacher nor the program matter much, do they?"

Again, where's the logic here? I didn't say the teacher didn't matter. But also, if you read the LEV blog, teachers are supposed to be social workers and doctors.

I'm not saying teachers aren't key (and never did but do keep trying, Reader), I'm saying that teachers CAN'T change what comes in the classroom and yes that has an impact on how much the teacher can do.

It's apples and oranges and all I'm saying is that we need to acknowledge that and not throw everything on teachers.

Maureen said...

Dorothy and reader, here is my take on (what I think is) your argument.

You can't discount the importance of a child's family influence if you are concerned about their academic achievement. "Good" teachers will create more "value" for kids who have had "good" parents, but "good" teachers will ALSO create more value for kids who have "bad" parents.

In addition, ALL kids would do better if their parents were "good." I.e., if we can put some focus into creating "good" parents, we will positively multiply the impact of the many "good" teachers we have.

(And reader, I, personally, would appreciate it if you could find something other than "DUH" to say when you disagree with someone's logic. That is one of the very few words I have forbidden my kids to say--I believe it reflects very poorly on the person who employs it. I think it sends a message of inferior intelligence (which conflicts with my impression of you dear reader!).

Josh Hayes said...

Even if one buys into the factory model of education, I think reader's point is, well, off-point.

Let's say you view a teacher at a particular grade as akin to the assembly-line worker -- parts arrive, pre-machined, ready to be fit together to pass on to the next station. A worker who's not good at this job does sloppy work, and the product delivered down the line is not up to snuff. Off with his or her head!

But in the school factory, the "parts" arrive from places outside the school. The factory worker has to take poorly-tooled, misshapen blobs of humanity, and try to fit them together into a "product" which can move successfully onto the next stage.

How on earth can one blame the worker when poorly-prepared parts are delivered to him or her? I have heard with my own ears parents explaining that their kid didn't do the homework because they, the parents, didn't think it was worthwhile.

How does a teacher, no matter how bad or good they are, use their mad teacher skillz to effect a change in that kid? And how do they recover from the fact, the FACT, that that kid will screw up their test scores?

I think reader's perspective betrays a common viewpoint, which is, kids aren't people. They don't have lives, they don't piss, or poop, or get distracted and depressed when their grilf or boyf is somehow distant -- kids perform on tests the way they perform that day, and the variety of factors involved in what that performance is beggars the imagination, from pop-tarts for breakfast, to gunplay in the neighborhood while they're trying to eat their goddam pop-tart, to, yes, broken fingernails. Stresses. Stressors. Different for every kid, and completely outside the ken, and the reach, of the teacher.

I'm not dissing you, reader, for not seeing this. I'm honestly astonished that you can't. Sometimes our view of our own lives prevents us from being able to understand something from outside that experience. Maybe that's what's happening here.

dan dempsey said...

A few thoughts from North Carolina.

reader said...
This comment has been removed by the author.
reader said...

Taking the factory model a little further Josh. If a child with 35% inferior parts comes in, we'd compare the performance of all the classrooms with 35% inferior parts, and see who does better on exit. I'm not sure where the bit about kids not being people comes in. The kids who come in... are the kids. They are the teacher's job, and that diversity they represent is supposedly a good thing, isn't it? I actually don't look at it as "inferior parts", but rather the complexity of the job.

Look, I totally get that tests scores, on their own is not a complete measure of anything. But equally ridiculous is the idea that the teacher is insignificant in student performance. That they are insignificant is simply a statement of the actual problem. I'm not saying this is a good plan. Once my kid is in the building... the teacher is the main influence on the student, plain and simple.

Actually the whole issue is a plague on both their houses. On MJG for offering up such a pittance to student peformance and quality without any hope of real benefit, and for reducing teaching to her own personal testing. But equally bad is the teacher's union for its blatant self interest, and intransigence. Their attitude is to never give an inch on anything... oh, unless it's for the new teachers. Then it's OK. This thing is voluntary, and so little as to hardly be worth quibbling over. Yet, quibble they do. The whole "merit bonus" would amount to essentially 1 latte per day. Do the Everyday Math.

Josh Hayes said...

All right, fair enough.

One quibble: the test-linked assessments are not voluntary for new teachers, which means that over time, every teacher will have to submit to them.

Melissa Westbrook said...

Thank you, Maureen, you expressed it much better than I did.

"But equally ridiculous is the idea that the teacher is insignificant in student performance."

Who said this on this blog? I have never heard it said anywhere, in any article or blog.

reader said...

Look Melissa, within the the parameters of the school system... the teacher is obviosly the thing that matters most.

To constantly deny it, or constantly state that "there's no way anybody can change predetermined outcomes"... well, that really just undermines the point of public education. The only fact is... nobody has bothered to change predetermined outcomes. We call it the achievement gap. While we might not like the measurement aspect of SERVE, encouraging teachers to teach where they are most needed, with financial incentives, can only seen as something positive.

dan dempsey said...

Read This, which contains:

Unless and until people are prepared to investigate the political dimension of the education catastrophe, we are never going to make any significant, permanent progress.

zb said...

To use the doctor example -- most doctors would probably argue that without clean water, their efforts to treat are going to have only minimal effect, and that further more, that differences between their treatment outcomes are going to depend very little on the quality of care they've offered. It's as valid to consider the issue for doctors as for teachers, and I think the fundamental model of correlation based measures for evaluating people is fundamentally flawed.

Say, for example, that we found that the education level of the parents accounted for 99% of the variance in student performance and the teacher for 1%. Then, does it make sense, as a society to say that all of our energy on improving the education (say, just the ability to read, to keep it simple) should be focused on improving the teacher (in some way we don't understand)? Or, should we be talking about improving the home environment?

We considered these issues when someone (probably teachers)brought up the fact that they simply couldn't teach a child who was hungry, and addressed it by giving the children food.

dan dempsey said...

It was said:

"within the the parameters of the school system... the teacher is obviously the thing that matters most."

I take issue with the idea that the most effective way to improve mathematics outcomes in the SPS is to improve the quality of teachers.

In my above posting I fault the current system. System improvement is far more important than Teacher Quality in the SPS at this time for Math and likely everything else as well.

To improve teacher math content knowledge at this time under the current parameters to the point where a significant improvement in student learning will occur is highly improbable.

The are a number of instructional materials and instructional practices that need to be changed to bring about significant improvement in mathematics but these changes will not happen. The current political governance and decision making process makes the intelligent application of relevant data impossible.

Charlie Mas said...

I'm all for rigorous evaluations for teachers. I also think that a teacher's work should be evident in student academic progress. But I don't think that we should measure that progress along a single dimension.

In addition, I think it is extraordinarily difficult to determine what elements of student academic progress are attributable to the teacher's talent, skill, and work. I don't think that the MAP score - or even the delta in the MAP score - is a reliable measure of anything meaningful relative to the quality of the teacher's work.

I recognize that in the SERVE proposal the student growth element of the evaluation would account for only 25% of the total evaluation, and the MAP score part of that might be even less, but I just don't see it as meaningful in any way. This is not what MAP measures and it cannot be interpreted to measure this.

Here's how I would like to see MAP scores used as part of a teacher evaluation:

1) Question: How did you use MAP scores to individualize instruction for your students?

2) Question: The MAP scores indicated that these students were not working at grade level. What did you do to accelerate their instruction and bring them to grade level?

3) Question: The MAP scores indicated that these students were working beyond grade level. What did you do to provide them with instruction at the frontier of their knowlege and skills?

As for measuring student progress attributable to the teacher's work, I don't have an alternative. I've yet to see anyone offer one. All of the value-added studies that I've seen that compare teachers to each other, such as the Tennessee one, presume that all classes are equal (or at least statistically equivalent). I can't accept that proposition. Perhaps if we had a large enough sample over multiple years and student-teacher assignment were random, then we could make that leap, but I don't see cause for it now. To think that in a class of 30 every student determines 3.3% of the outcome and to see that the variance in outcomes isn't much more than a few points, puts WAY too much weight on the distribution of students instead of the distribution of talent.

reader said...

Do we remove doctors from areas with dirty water? Do we say, oh well, no need to have good doctors? Anybod will do? Do we provide bad doctors to smokers? Why bother? If you're a smoker, you deserve what you get. Don't you?

Look, here are the issues.

1) We've got a cadre of mediocre teachers. Anybody who has had a kid or more, will see the high rate of teacher mediocrity. Sure, there are some good ones. But, mostly not really too good. And, some downright bad. They are entrenched in their maintaining status quo and self-interest. Ever met Olga? She's the picture of inflexible self-interest. (point MJG)

2) We've got a lot students not doing too well, and teachers with no plan to address that. No ideas, no responsibility. The students have the right to be there, no matter how many inferior parts they are comprised of. Not true for the teachers. Teacher's seem to think that their right to a job is equivalent to a student's right to an education. Completely untrue. (point MJG)

3) We've got lifetime contracts with no basis in performance. Not only are these maximally expensive, they create perverse incentives. (point MJG)

3) We've got a proposal for a simple minded test that attempts to put some accountability into the system. (point teacher's union)

4) We've got a system of standardized testing that creates all sorts of perverse incentives. They are anti-teamwork, anti-student, anti-challenged student, and promote teach-to-the-test. (point teacher's union)

reader said...

Dan, what sort of math do you think they use at the private schools? The high performing private schools? Let me give you a clue. It isn't
"direct instruction" a 100% scripted plan from SRA (huge corporation)... the thing you love so much,nor Saxon Math, nor Singapore. It's inquiry based, and it's discovery. When you learn from inquiry, you learn more. And, it's about the teachers.

My kids have been subjected to "direct instruction" and all the other crap you propose as good. It isn't. And, it doesn't take you far.

When the math teacher is no good... blame the book. It isn't about the book.

Charlie Mas said...

And let's be really clear about the parent/teacher attribution thing.

I absolutely agree that the teacher effectiveness is the number one SCHOOL-BASED determinant of student academic growth.

However, overall, the primary determinant of student academic achievement is, without a doubt, the active involvement in the student's education by an adult in the student's home.

The only time I correct people on this - and I know that I've had to do it a lot - is when they don't mention that the teacher is paramount only within the conscribed sphere of school-based determinants and make it seem that the teacher is the primary determinant overall.

Overall, the home-based influences overwhelm the school-based influences.

I will be delighted to find an objective measure of student progress that is attributable to the teacher.

Here's the funny thing. Most of the people who want these measures to find and dismiss the ineffective teachers also claim that everyone knows who those ineffective teachers are. So even they will admit that we don't need these measures to identify them. So what do we need them for?

reader said...

We need them (the measures of effectiveness) to break the codification of lifetime appointments with no accountability.

Funny how the unions are SOOOO anti-standardized testing. (with good reason). But when it comes to hiring, they want nothing BUT standardized testing. Isn't that ironic? All interviews must be scripted to the max, and all designed by the teacher's union. All interview questions must be indentical. Expected "right" answers are scored. No discussion is allowed. No probing deeply into any given candidate, their background, their special talents, their unique character, their fit for the job. No discretion can be given to the principal or other hiring. And under NO circumstance would an interview ever include teaching a sample class, for a class period, or even a day, in front of real live kids. NEVER! Might hire the best candidate in that case. They sure don't want that to happen.

When it comes to hiring.... the union is only too happy to have widgets. Why then cry when they are treated like widgets for promotion or firing? The hiring process is designed to normalize mediocrity, and to expect it.

zb said...

"I absolutely agree that the teacher effectiveness is the number one SCHOOL-BASED determinant of student academic growth"

I don't. I think it's peer group and fit (and by fit, I mean the interaction between the teacher and the students). I'm just not seeing good evidence to convince me that teachers are "the most important school-based factor in education."

"Do we remove doctors from areas with dirty water?"

No, but we don't fire doctors (or refuse to reimburse them in medicare) based on the proportion of diabetic patients who control their blood sugar, either.

The problems evaluating performance are very similar, and I'd be more willing to consider these numerical measures for teachers if we were considering them for doctors, too.

seattle citizen said...

Here's a scenario:

Student has six classes:

Math, Science, Art, History, Remedial Reading, and Language Arts

Additionally, student is on track team and debate.

All involve reading, all involve the teaching of reading skills and/or metacognative skills that benefit reading.

In the evening, the student attends a reading tutorial program at Kumon or somesuch.

When being tucked in (okay, it's a middle or high schooler, and that would be kinda strange, but bear with me) the student's grandmother reads stories.

Student scores a...Level 3 on HSPE and a...221 on MAP.

Who is responsible? Who should be rewarded or penalized?

seattle citizen said...

Of course I meant HSPE and MAP scores in Reading in the above scenario.

And I'd add one more player:

Imagine that the student only gave 50% of their attention per day to the other players, and that 50% attention varied day by day regarding who it was given to.

Imagine also that in the fall the student is distracted blows off the Fall MAP. In the winter the student doesn't like the teacher, but still works hard....kind of, and shows an increase of, oh, let's say any old arbitrary number...three points. In the spring the student likes the teacher, and works very, very hard on the test and shows an increase of ten points!

A) NOW who gets the credit (or discredit) for this child's score on the MAP (or WASL or whatever)?

B) does this score tell us what the student learned, and/or how well they learned it?