Bill Gates, Part 1: How Much Will He Spend Before He Admits He Doesn't Have the Answers for Public Ed?

The latest about the Gates Foundation is the Rand Foundation report on the Foundation's efforts in teacher evaluation around using test scores.  What was the Gates Foundation doing?

The Intensive Partnerships for Effective Teaching initiative, designed and funded by the Bill & Melinda Gates Foundation, was a multiyear effort to dramatically improve student outcomes by increasing students' access to effective teaching. Participating sites adopted measures of teaching effectiveness (TE) that included both a teacher's contribution to growth in student achievement and his or her teaching practices assessed with a structured observation rubric.

Beginning in 2009–2010, three school districts — Hillsborough County Public Schools (HCPS) in Florida; Memphis City Schools (MCS) in Tennessee (which merged with Shelby County Schools, or SCS, during the initiative); and Pittsburgh Public Schools (PPS) in Pennsylvania — and four charter management organizations (CMOs) — Alliance College-Ready Public Schools, Aspire Public Schools, Green Dot Public Schools, and Partnerships to Uplift Communities (PUC) Schools — participated in the Intensive Partnerships initiative. RAND and the American Institutes for Research conducted a six-year evaluation of the initiative, documenting the policies and practices each site enacted and their effects on student outcomes. This is the final evaluation report.

The cost was about $575M of which the Foundation paid half.  Districts kicked in the rest.  And, this was during the Obama/Duncan era so they certainly were on-board with Race to the Top funds.  From the assessment by Valerie Strauss of the Washington Post:

From the start, critics had warned about using a standardized test designed for one purpose to evaluate something else — a practice frowned upon in the assessment world.

In a desperate attempt to evaluate all teachers on tested subjects — reading and math — some of the systems wound up evaluating teachers on subjects they didn’t teach or on students they didn’t have. Some major organizations publicly questioned them, including the American Statistical Association, the largest organization in the United States representing statisticians and related professionals. And so did the Board on Testing and Assessment of the National Research Council.

The key finding?

Sites implemented new measures of teaching effectiveness and modified personnel policies accordingly but did not achieve their goals for students.

According to Rand, why was this initiative not successful?

There are several possible reasons that the initiative failed to produce the desired dramatic improvement in outcomes across all years: 

• incomplete implementation of the key policies and practices; 
• the influence of external factors, such as state-level policy changes during the Intensive Partnerships initiative; 
• insufficient time for effects to appear; 
• a flawed theory of action; 
• or a combination of these factors. 

Recommendations (bold mine):

• Reformers should not underestimate the resistance that could arise if changes to teacher-evaluation systems have major negative consequences for staff employment.
• A near-exclusive focus on TE might be insufficient to dramatically improve student outcomes. Many other factors might need to be addressed, ranging from early childhood education, to students' social and emotional competencies, to the school learning environment, to family support. Dramatic improvement in outcomes, particularly for LIM students, will likely require attention to many of these factors as well.
• In change efforts such as this, it is important to measure the extent to which each of the new policies and procedures is implemented in order to understand how the specific elements of the reform relate to outcomes.

That statement in bold is key for ALL ed reform.  There is not just one single thing that will change the trajectory for students who struggle.  (Well, if we had a guaranteed income so there are no children living in poverty, maybe.  And no, we don't have to solve poverty to help kids but a good teacher can't do it alone.)

But what about more money? From the Rand brief:

Some compensation and career-ladder policies were enacted to retain effective teachers, but they were not as extensive as envisioned, did not always follow best practices, and were not necessarily incentives about which teachers cared.

All seven participating sites implemented effectiveness-based compensation reforms, which varied in terms of timing, eligibility criteria, dollar amounts, and the proportion of teachers earning additional compensation. Teachers generally endorsed the idea of additional compensation for outstanding teaching, but (except in two of the CMOs) most reported that their sites’ compensation systems did not motivate them to improve their teaching.

A favorite saying in the educational measurement community is that one does not fatten a hog by weighing it. In the end, the sites were better at implementing measures of teaching effectiveness than at using them to improve student outcomes.

So much for greedy teachers and their unions.