Gates Foundation Flash: It Takes "Multiple Measures" to Judge Educators
From Ed Week:
Student feedback, test-score growth calculations, and observations of practice appear to pick up different but complementary information that, combined, can provide a balanced and accurate picture of teacher performance, according to research released today by the Bill &Melinda Gates Foundation.
The $45 million study, in progress since 2009, is one of the largest and most extensive research projects ever undertaken on the question of how to identify and measure high-quality teaching. It involved some 3,000 teachers in six districts: Charlotte-Mecklenberg, N.C.; Dallas; Denver; Hillsboro County, Fla.; Memphis, Tenn.; and New York City.
To the amazement of all:
Basing more than half a teacher’s evaluation on test-score-based measures of student achievement seemed to compromise it, the researchers also found.
Another piece suggests that teachers should be observed by more than one person to ensure that observations are reliable.
I'm just gobsmacked.
From the Times via the AP:
Several districts involved in the research acknowledged that student surveys were the most controversial part of the process, and some, like Hillsboro County Public Schools in Florida, have opted to leave them out of the mix when scoring teachers.
Jean Clements, president of the Hillsboro Classroom Teachers Association, said her district decided the results of student surveys, which ask questions like "do you feel challenged to do your best work," may not be trusted by teachers.
The researchers found, however, that student surveys help teachers improve their practice because those results evoke the most emotions.
You mean because teachers hearing from students about what works and doesn't work, what motivates and moves them might help their teaching? Good to know.
Student feedback, test-score growth calculations, and observations of practice appear to pick up different but complementary information that, combined, can provide a balanced and accurate picture of teacher performance, according to research released today by the Bill &Melinda Gates Foundation.
The $45 million study, in progress since 2009, is one of the largest and most extensive research projects ever undertaken on the question of how to identify and measure high-quality teaching. It involved some 3,000 teachers in six districts: Charlotte-Mecklenberg, N.C.; Dallas; Denver; Hillsboro County, Fla.; Memphis, Tenn.; and New York City.
To the amazement of all:
Basing more than half a teacher’s evaluation on test-score-based measures of student achievement seemed to compromise it, the researchers also found.
Another piece suggests that teachers should be observed by more than one person to ensure that observations are reliable.
I'm just gobsmacked.
From the Times via the AP:
Several districts involved in the research acknowledged that student surveys were the most controversial part of the process, and some, like Hillsboro County Public Schools in Florida, have opted to leave them out of the mix when scoring teachers.
Jean Clements, president of the Hillsboro Classroom Teachers Association, said her district decided the results of student surveys, which ask questions like "do you feel challenged to do your best work," may not be trusted by teachers.
The researchers found, however, that student surveys help teachers improve their practice because those results evoke the most emotions.
You mean because teachers hearing from students about what works and doesn't work, what motivates and moves them might help their teaching? Good to know.
Comments
Ever heard of the Golden Rule? S/he with the Gold Makes The Rules.
We building serfs don't allocate the re$ource$, or make the big decisions which impact the individual schools, but, the Golden Boyz of Billie gotta drop big coin proving that the building serfs are the problem.
And we nobodies must pretend this is a debate about policy and about ideas, when it is just a power grab by yuppie scum... and if we call them what they are, and if we call it what it is, then we're lowering the terms of debate, and we're turning people off, and we're going ... TO LOSE!!!
Ummmm ... aren't the liars doing pretty good with us shuffling, bowing and scraping, trying to not upset The Lord And His Lady?
They couldn't have made a worse waste of the of the money by getting the $46,000,000 in bundles of hundreds & dropping it out of the sky into some Golden Triangle drug lord's compound.
WOW! Liars have to fess up to
BeingLiars!
I am reminded of the Seattle School Board acting on the revising of how Teachers Would be Evaluated ... while adopting yet another lousy Reform Math program. (Note the SEA had no objections to either).
Hattie makes it clear that the best way to improve student performance of a school is to empower teachers as a group in each school to plan the path to improvement and allow them to manage and adjust that plan. Teachers would do that through the intelligent application of relevant data.
Unfortunately supposed "Professional Learning Communities" have become centered on top down indoctrination of supposed "Best Practices" in far too many schools.
The Gates Foundation was the funder of the original meetings held in secret to develop the Common Core State Standards, which are unlikely to be used to attain a maximum positive effect. In my current school rational thought has been suspended. Although my school is among the worse performing in Nevada especially so in mathematics, we are waiting for the Common Core to improve us.
Side note: for WA State Schools
Pass rate on 8th grade math MSP testing in 2012
= 67.2% (40.9%) Statewide NonLow (Low Income)
8th Math Pass rate for American Indians
= 30.3% statewide
= 33.3% Seattle
= 32.1% Tacoma
= 63.2% Puyallup School District
= 6.0% Chief Leschi School (near Fife)
= 5.0% Lummi Tribal Elementary School
= 13.2% Marysville SD
= 11.1% Totem Middle School** (in MSD)
= 13.5% Mount Adams SD (Yakima Rez)
= 14.3% Cape Flattery SD
= 42.9% North Thurston Public Schools (Lacey)
= 20.0% Wa He Lut Indian (Nisqually Rez 7th grade)
---------------------------------
At the 4th grade level in Wellpinit WA on the Spokane Rez MSP Math Pass rate
= 26.1% for Indian Students at
Wellpinit Elementary**
At the 4th grade level in Marysville SD
= 0.0% Tulalip Elementary School**
** a State School Improvement Grant school for two years at the time of testing.
= 4.3% Mount Adams SD on Yakima Reservation
===========================
Schools especially in remote locations with high poverty and high Indian student populations have particular challenges .... yet the latest one size fits all is coming to supposedly improve the situation in yet another one size fits all. Of course it will be applied in the Top Down fashion at most locations.
In Indian Country the CCSS is just the latest debacle. The CCSS debacle was co-sponsored by the Bill and Melinda Gates Foundation.
I would add that there are many students who have trouble with these reform textbooks, not just disadvantaged students. Outside tutoring is no substitute for a sound curriculum. A new approach could help many students and reduce the need for remedial math in college.
S parent
Combined Measures Better at Gauging Teacher Effectiveness, Study Finds
Here is Jay Greene's excellent analysis of what this BMGF study really says: Understanding the Gates Foundation’s Measuring Effective Teachers Project
Greene's conclusion:
So, rather than having “figured out what makes a good teacher” the Gates Foundation has learned very little in this project about effective teaching practices. The project was an expensive flop. Let’s not compound the error by adopting this expensive flop as the basis for centrally imposed, mechanistic teacher evaluation systems nationwide.
"It took them 3 years and how much money to figure that one out?"
Cyclops
"The project was an expensive flop."
Let's wait and see if the new Seattle Superintendent or the School Board seize upon the findings as a basis to roll back MAP testing and other overreaches that our teachers and schoolchildren are living with from past decisions. If they do, I would respectfully disagree. It would be a major victory. And while we can cynically say it was all about money buying political cover, that is the way the game is played and always has, since our founding. The District may be better off with this expensive flop than had it never been underwritten, but we will never know, will we?
- NE Dad
I have personally experienced many instances of counter-intuitive results from systematic study of topics that "everyone knows" are a particular way. So I don't buy that studies themselves are a waste of time---that's basically an anti-intellectual position, which is ironic given the topic.
Is it the way these studies were done? The critiques of the methodology I have seen complain that the correlations of teacher observation with test scores are not strong. Fine, that means that student test scores are the sole measure for teacher effectiveness? I keep seeing critics of "reform" citing test scores as proof that such-and-such approach won't work. Way to buy in.
Perhaps the issue is one of focus? That we should spend our time elsewhere because teacher effectiveness is not an issue? That's news to me. I've got multiple counterexamples I can give you right now. Those crappy teachers don't matter?
Maybe some are so tired of hearing about teacher effectivess from people with an agenda that we can't talk about it anymore, whether it's a legit topic or not?
I sense that some of you are saying "we already know what Gates is spending all that money to find out." At which point I have to ask what those common-sense everyone-agrees obvious answers are.
This echo chamber stuff is just getting to me I guess.
RollerCoasterFabio
If, as I believe it is, the pressure to make teachers more accountable according to test scores rubrics used only by principals is more about ideology and politics than it is about effective teaching, then nothing will come of this research. It's really up to the union now to take this research and use it to get supporting a broader, fairer, and more constructive way of holding teachers--and principals--accountable.
"It could if the union uses it effectively, but if it doesn't it's because the whole business of teachers evaluations so far has been about imposing a top-down model having more to do with asserting administrative control and less to do with what really would be helpful in making teachers more effective."
Exactly spot on correct.
BMGF is focused on tools that use Centralized Top Down One Size fits all control. BMGF pushes an agenda. As Jay Greene shows BMGF purposely misinterprets data to produce findings that support a predetermined agenda.
This is not about making teachers more effective.
But to have spent so much money only to say they figured out you need multiple measures? It seems sad because it is common sense.
No one gets evaluated on one point at their job and so why should teachers?
What would be good to know are the best ways for peer teachers to review work and how to get the best input from students.
Absolutely. But how do you suppose we should figure out what those 'best ways' are? Or whether those two things are even valuable?
If it involves doing a 'study' I suspect those concerned about a top-down model will be just as worried. I understand why, but you can't have your cake (freedom from administrative control) and eat it too (consistent use of best practices across a large district).
RollerCoasterFabio
They should have used that $46M to figure that out.
They are part of the overall solution. But all the big ed reformers make them priority no. 1, then the Peter Principle makes them the be all and end all. Remember, we are dealing with human beings, after all.
Hence the objection is not against testing, per se. It's against too much testing, and especially "high-stakes" testing.
WSDWG