Bill in the Leg, Column in the Times
There must be a teacher evaluation bill in the legislature because there is a guest column about something similar in the Seattle Times. "It's hard to overestimate the value of a good teacher" by Nicholas Kristof is all about the false concept of "teacher quality". It is based on the most ridiculously and obviously flawed analysis possible, the circular logic that great teachers can be identified by how their students' test scores rise and the proof of their greatness can be found in the rise of their students' test scores. We might as well identify great teachers by how much taller their students grow and then use the increase in their students' height as the proof of their great teaching. The truth is that "teacher quality" has only an incremental influence on student test scores and therefore student test scores are an extraordinarily poor measure of teacher quality.
Comments
--seeking more info
-seeking more info too
And of course, from Michelle Rhee's lips to Rodney Tom's ears and mouth, it's all "for the kids" and about "giving kids a voice." Please.
It's called union-busting 101. Prepare for further diminution of wages, and thereby quality, of teachers and staff in the continuing race to the bottom for the middle class.
To hear Pettigrew and Tom speak is to witness the ignorant proclamations of white elephant recipients, passionately trading away the keys to the store for a jar of snake oil. It's enough to make me puke. More democrats and supposed liberals selling out the the people who need help the most. Sad. WSDWG
Read up on what happens when state passes law like this and then try to implement them:
http://www.nytimes.com/2011/11/07/education/
tennessees-rules-on-teacher-evaluations-
bring-frustration.html?_r=1
It is as if we have traded our good common sense for jargons and knee jerk reactions. What we will get is more administrators, more testings, more bureaucracy, more computers, more studies and more reports and all of this takes MONEY. Lots of it. But not in the classroms.
My kids will spend more time taking more tests and less time learning in their classes, and less time learning things beyond what will be on these standardized tests. My kids will not really benefit from these tests because unlike a biology unit test or an essay graded with comments, these tests just give them a number score. My biggest fear (as I'm seeing this already) is that these tests will determine what kids learn. These tests will set the de facto curriculum. My kids will skim through courses with basic facts, but not how to put facts, context, and critical thinking together to really learn a subject matter. And of course, less room for their creativity and curiosity to be engaged as these tests don't measure for that.
(By the way,"Seeking info", anyone with an internet access can research what you asked for. It is all there
in NYT, Washington Post eduction, and Education Week, Shanker Institute sites. Read comments attached to many of these articles. Read about the implementation of these laws and how the rush to get them through fail to address the implementation portion. Read about what happens when states fail to meet the goals they set up in these bills.)
Upset parent
I almost can't believe this could be true (any economist or policy analyst worth their salt would see this in a minute and Eric Hanushek is a good economist), but I've never found a study that clearly defines how teacher quality independent of test scores has been defined. Has anyone else?
(If I were doing the study, I would have a very expensive detailed system of teacher evaluation (using peer and administrative evaluators over a long period of time) and then go back and look at the relationship of those independent scores to students' test score gains. I have never come across a study that seemed to do it that way.)
Ok, I started skimming the study (thanks mirmac!) and they seem to be acknowledging that people might think that using test scores to define teacher value added might be an issue, so they went through a process to show that it's ok in terms of the independence of the variables. What they don't seem to be adressing is the issue as to whether or not teacher quality has anything to do with test scores (they are talking about quality as teacher value added, I would argue, they are not necessarily the same thing.)
I'll try to make time to review the study, but I haven't really done this stuff in a long time. Could someone more current take a look?
Shanker blog on study
I actually can see how this would apply across a very large sample over time. What I can't see (still skimming!) is how on earth you could ever think you could apply this to any individual teacher at any given point in time. Things that work with large numbers totally fall apart for individual data points (i.e., human beings!).
http://shankerblog.org/?p=4708
For the generalists, Matt DiCarlo discusses the study and relates that to policies. His quotes:
"The fact that teachers matter is not in dispute. The issues have always been how to measure teacher effectiveness at the individual-level and, more importantly, whether and how it can be improved overall."
AND
"On the other hand, this report’s findings do not really address important questions about the proper role for these estimates in measuring teacher “quality” at the individual level (as previously discussed here), particularly the critical details (e.g., the type of model used, addressing random error) that many states and districts using these estimates seem to be ignoring. Nor do they assess the appropriate relative role of alternative measures, such as principal observations, which provide important information about teacher effectiveness not captured by growth model estimates.
Most importantly, the results do not really speak directly to how teacher quality is best improved, except insofar as it adds to the body of compelling evidence that teachers are important and that successful methods for improving teacher quality – if and when they are identified and implemented – could yield benefits for a broad range of outcomes over the long-term.***
Most of the popular current proposals for quality improvement – such as performance pay, better recruitment, and improvements in teacher preparation and development – are still works in progress, and most have either scant or no evidence as to how (and whether) they can serve to improve educational outcomes (test-based and otherwise). The results presented by Chetty, Friedman and Rockoff do not tell us much, if anything, about the desirability (or lack thereof) of any of these interventions."
upset parent
http://shankerblog.org/?p=4708
For the generalists, Matt DiCarlo discusses the study and relates that to policies. His quotes:
"The fact that teachers matter is not in dispute. The issues have always been how to measure teacher effectiveness at the individual-level and, more importantly, whether and how it can be improved overall."
AND
"On the other hand, this report’s findings do not really address important questions about the proper role for these estimates in measuring teacher “quality” at the individual level (as previously discussed here), particularly the critical details (e.g., the type of model used, addressing random error) that many states and districts using these estimates seem to be ignoring. Nor do they assess the appropriate relative role of alternative measures, such as principal observations, which provide important information about teacher effectiveness not captured by growth model estimates.
Most importantly, the results do not really speak directly to how teacher quality is best improved, except insofar as it adds to the body of compelling evidence that teachers are important and that successful methods for improving teacher quality – if and when they are identified and implemented – could yield benefits for a broad range of outcomes over the long-term.***
Most of the popular current proposals for quality improvement – such as performance pay, better recruitment, and improvements in teacher preparation and development – are still works in progress, and most have either scant or no evidence as to how (and whether) they can serve to improve educational outcomes (test-based and otherwise). The results presented by Chetty, Friedman and Rockoff do not tell us much, if anything, about the desirability (or lack thereof) of any of these interventions."
upset parent
I have three issues with the latest assault on public schools in legislation dropped yesterday.
Other states that have had charter schools for years have had a lot of bad experience with them, all sorts of types of corruption can and does occur with this "innovative" way to privatize public education. I was worried that pro-charter folk would craft a bill thinking that they would thwart this problem but not succeed; Pandora's box would be opened. However this current legislation doesn't even attempt to protect Washington from the abuses of charters we have seen in other states.
Why is the legislature the place to codify how to evaluate teachers? Do you also codify how to evaluate fire fighters and park rangers? Yes, this is work that needs to be done and we are doing it. Seattle has a CBA that is piloting the new evaluation framework. But we are only halfway through the CBA. Do we have results yet, analysis of how it is working, does it have the intended effect on education? No, we do not have that analysis yet. But our Federal TIF Grant requires outside evaluation and we have a contract with a firm to do just this work. Doesn't it seem more reasonable to get this feedback before jumping in whole hog? Passing legislation mandating wholesale changes to teacher evaluation without better knowledge of what works is dangerous. Additionally it's an unfunded mandate.
That leads me to my third point. Why in the world are you folks discussing such expensive new initiatives when you don't have any money? It seems criminal to me for any legislator to be pushing to bring in all the extra bureaucratic layers at both the state and district level when there is not one extra dime in the budget to pay for it. The luxury of debating the merits of new costly initiatives like this should be considered only when the state has the luxury of being able to budget appropriately for them.
Respectfully,
Dorothy Neville
Constituent, 46th LD, member 46th DEMS,
Former PTSA member and board member -- Treasurer of Roosevelt HS PTSA
Golden Acorn award recipient.
Please note that the Washington State PTSA with their pro-charter agenda does NOT speak for me.
What about control for factors beyond patient's individual health history such as the institution the nurse or doctor works in. Is it a good clinic or hospital, rural vs. urban, patient to staff ratio, up to date facilities, number of mediclal/surgical specialties, level of trauma/medical care, nonsocomial infection rates, what best safety practices are in place, is there good communication, inter-disciplinary team work in place, etc.?
What about the person you are evaluating? How much does the level of education, training, and years of clinical practices matter?
Complicated you bet. So you need to be careful about these quick fixes, quick answer solutions. They provide and measure less than what you think. Sometimes, they serve more to hinder than encourage best practices.
Upset parent
I just wrote a message to Dorothy N. congratulating her on her letter to David Frockt..........and I also in the same message wrote a letter to Eric Pettigrew. I know him and wanted this to be a personal plea and disagreement with his proposed legislation on charter schools requesting he withdraw and reconsider his position. I was writing from discussions we had last night at the UWAA MAP Board meeting. He has participated in UWAA MAP's advocacy and events to insure equity for students for many years, and we are angry, disappointed and confused. It was a good letter.
Could someone help me to retrieve this post as the web genie said I was over the time allotted or something like that.
Thank you,
Carol
First, I am astonished by the circular logic employed in these studies. They identify the "effective" teachers by using student test scores and then use those same student test scores to prove the effectiveness of the teachers.
I shouldn't have to point out the self-fulfilling flaw of this process.
What's missing, of course, is any sort of attribution analysis that quantifies the teacher's role in determining student test scores.
The data suggests that the bulk of student academic outcomes is determined by the student's home - the affluence, the education level of the primary caregiver, culture, and the active involvement in the student's education by an adult in the child's home.
There are a number of school-based factors other than the teacher. There are the curriculum and materials. There's funding, class size, and supporting resources like the library, technology, and instructional specialists. There are things that aren't even in the classroom. Think of the folks who want to give credit for rising scores to the principal, the Executive Director of Schools, the Chief Academic Officer, the superintendent, and the Board.
So after the home-based determinants and after all of the other school-based determinants of student academic outcomes are removed, there is very little of the result that is available for attribution to the teacher.
On top of that we have a lot of contradictory data. Among the thirty students in a classroom we are likely to see a broad distribution of outcomes. If a great teacher has the kind of impact that the Education Reform movement would have us believe they have, then why would there be any under-performing students in the great teacher's class? But there they are.
Then there is the simple human truth that every teacher-student relationship is different. The teacher who is great for Student A may be dreadful for Student B. Is the teacher great or dreadful?
So the whole discussion of teacher quality is something of a fiction and it is inherently false for anyone to pretend to have identified the effective teachers or have concluded anything about their impact.
It's possible that some of these questions can be resolved, but let's not give the teacher either all of the credit or blame for student outcomes. That's just goofy.
The only thing goofier is giving the credit or blame to someone in the district's central administration.
Another point about this charter distraction that I want to make with our lege is the fact the WA voters voted for smaller class sizes back in 2000 with Initiative 728, and by a significant margin -- 72 percent -- (see Ballotpedia), but our state has yet to make good on that. (I've heard the money for it was diverted elsewhere.)
Whereas WA voters have voted NO to charters three times in the last 15 years or so.
So why the heck are our reps contemplating pushing something WA voters have clearly said they don't want -- multiple times -- and ignoring what voters have demanded?
Whose will are they representing?
I would argue that smaller classes are a clear example of what WA voters do want.
Our legislators should not be pursuing another distraction while they have yet to make good on existing demands and obligations like class size reduction and fully funding our schools. -- Both of which would help all kids and any "gaps" in the system.
My letter to my elected officials will contain the "If Charters were the magic solution, we'd have seen reproducible magic by now."
As it is currently reported that WA State has "No Planning Activity Reported" in regard to CCSS and Teacher Evaluation Systems ..... Here comes Eric P to the rescue.
Fellow 46ther
It is called paternalism.
These individuals want to hold teachers accountable. So, can we hold these individuals accountable and throw them out of office for failing to fund our schools? How about paying for full day K? How about paying for 180 days of school? How about paying for 6 periods of high school? What is good for the goose should be good for the gander. My guess, these chowder heads will pull more dollars out of our schools during this legislative period- and fail to fulfill our consititutional duty. But, they have plenty of time to attempt charter schools.
SueP, yes on the I728 comparison. Although to be accurate, we did get I728 money but it has dried up with no return in sight.
Washington receives a not-so-gentlemanly "C" and ranks No. 38 among 50 states in the new "Quality Counts" comparison of states' commitment to schools, put out annually by Education Week magazine.
Report: Wash. gets an 'F' in education spending
The Education Week study ranks Washington 42nd in per-pupil spending and 44th in state expenditures as a percent of state taxable resources.
..... Lack of adequate support is the reason for other low grades. Washington gets a "D-minus" for teacher professional development. The state fails to finance professional development for all school district, lacks a formal teacher induction and fails in having a monitoring program for all new teachers.
==============
It appears Mr. Pettigrew needs to point the finger at himself .... if he wishes to find and evaluate for accountability.
My problems with this teacher-quality legislation are twofold. First, apart from the foolishness of linking quality to teacher performance to student performance on test scores, it assumes principals are in a position to judge teacher quality. Some are, but most are not.
Principals are by and large people who do not like to teach or who were no good at it, and that's why they went into administration. There are good principals who clearly do not fit this description, but system-wide too many of them are middle managers more than they are truly educators, and the idea of putting the whole teacher evaluation process on their shoulders is terribly misguided.
If we had a system in which principals were in fact "master teachers", then we'd have a different situation. I hope over time we move toward that, but in the mean time the union in partnership with the school board and new supe should develop a more collaborative evaluation system in which all mature stakeholders play a role in evaluating not just teachers, but principals, too. Principals need to know they serve their learning communities, not the bureaucrats downtown.
Second, of course, really bad teachers should be fired, but everybody knows who the really bad teachers are. The real challenge is improving the quality of instruction given by the average teacher. And this perform-or-be-fired mentality doesn't solve the problem at that level. If you fire one mediocre teacher, more than likely you will just replace him with another.
If we're serious about improving teacher quality, we need to do something along the lines of what Finland has done. The legislators yesterday said that every child deserves a "great teacher". It's a silly statement. No one can guarantee 'greatness', but you can guarantee a certain level of competency and quality. Real improvements along those mines, however, are not going to happen without a serious commitment, both of political will and economic resources, to making a career in teaching a lot more attractive than it is now.
But corporate ed types are pushing in the opposite direction. They are making conditions so bad for teachers, that pretty soon you will have have to be a hero of self-sacrifice or a moron to want to be one. There are few of the first, and in the long run the teaching corps will be dominated by the latter if the corporate ed types get their way. The worse the classroom teachers are, the more effectively they can make the case for the necessity of online education.
The legislature has contributed to making teaching a lot less attractive. Pettigrew and Tom continue marching in the increasingly unattractive direction.
http://www.joycefdn.org/resources/content/9/3/3/documents/Teacher-Eval-Report-FINAL.pdf
SEA member
Because lower class size costs A LOT, charters don't. Class size reductions have to be a huge cost requiring both more teacher costs and more school buildings. Voters, as per usual, are all in favor of something for nothing - but that isn't possible. We are pratically the only state left that doesn't have charters. And, it is the strong union lobby that has barely kept it at bay so far.
-reader
The study showed something different: that having a better teacher even for one year correlates strongly with improved test scores for students -- and that the end result over several years is improved student achievement in school, higher income and more cautious behavior (reduced teen pregnancy, for example). I think it is very hard to argue that teacher quality can't be measured when it's tied to real-world student achievement and experience over many years, even into adulthood.
The study remarkably controlled for income and other factors that often divide students into defined groups of high or low achievement. That is, whether the school has high- or low-income students and whether the students are high- or low-achieving on a day-to-day basis, teacher quality makes a difference over the long-term -- and that quality can be reflected by measuring test-score improvement.
The research shows that having a high-quality teacher instead of an average teacher -- even for one year -- improves the outcomes of students. And having an average teacher instead of a poor teacher -- even for one year -- improves the outcomes of students.
The study's researchers do propose solutions: use test scores to measure student achievement and, therefore, teacher quality. Replace poor teachers sooner, rather than later. And invest in average teachers with training to raise their skill level and quality.
Could you tell us how the study showed that a student's behavior/grades/outcomes years in the future could be tied to what a teacher did or didn't do years in the past? I don't get it. Does this study purport to show that the actions/inactions of Teacher X in 2002 resulted in a college/dropout, or "caution"/recklessness in 2010?
Seems like a stretch....
So a kid's pregnancy in 2010 can be attributed to their 2nd period teacher in 2004? Hmmm....
I read the Chicago report and sure thank "SEA member" for posting that link.
You wrote:
I think it is very hard to argue that teacher quality can't be measured when it's tied to real-world student achievement and experience over many years, even into adulthood.
I am going to attempt to argue against that. I've taught in a large number of different types of schools in four states beginning in 1968.
The idea that a consistency of improved test results comes from teacher quality may often be true. I do not see that as being the case universally however. There is a sizable contribution coming from "other factors". .... I attempted to teach a class of 40 low achieving 7th graders with the District mandated "at grade level" materials. The class included three children who spoke no English. Several students had fathers in prison.
Deming sees a significant number of a systems problems as due to other factors than employee inadequacy. If you notice in the Chicago study that historically 0.3% of Chicago teachers were rated as Unsatisfactory .... and in this two year study 3% were rated as Unsatisfactory..... Perhaps 97% of a schools deficiencies are due to other factors.
=====
Deceptively SB 5914 says:
Sec. 101. This act may be known and cited as "the excellent teachers for every student act".
This piece from the Columbian makes in know this is a lot more about teacher pay.
Immediate reduction in board-certification bonuses and other changes would net $45 million savings in the 2011-13 budget period, Zarelli said.
==================
In particular, these policies support practices with a track record of closing the achievement gap. This is done by:
SB 5914
(1) Ensuring that teachers who do the best work are the ones who keep their jobs when budgets need to be cut, by basing reduction in force policies on the evaluations the legislature has outlined for measuring teacher performance. Since the loss of teachers through layoffs already impacts student learning, there is an urgent need to conduct layoffs in a way that retains the most effective teachers.........
(2) Empowering principals and teachers with autonomy in challenging ......
(3) Recognizing that for the fair evaluation of a principal based on the criteria outlined by the legislature, specifically that principals should be evaluated on creating a school culture that promotes the ongoing improvement of learning and teaching and managing .....
(4) Increasing the authority of principals to terminate teachers upon a finding that, after multiple years of unsuccessful improvement following training and mentoring, .....
(5) Reforming the statewide salary allocation schedule and national board bonuses to ensure that state appropriations are rationally allocated using an evidence-based pay schedule ......
=========
This bill has been kickin' around since the regular session in 2011 in the Spring.
Back then the Supreme Court had not yet ruled on full funding of education .... I suggest that Rodney Tom re-evaluate.... if in fact he does wish to close achievement gaps and get closer to the goal of an excellent teacher for every student ..... currently he is on track to have a lot more frustrated teachers who feel unappreciated.
The bi-annual savings that were predicted for the general fund:
2011-2013 => $58 million
2013-2015 => $45 million
2015-2017 => $56 million
(4) All collective bargaining agreements and other contracts entered into between a school district and an employee bargaining unit or an employee after the effective date of this section, as well as bargaining agreements existing on the effective date of this section, but renewed or extended after the effective date of this section, must be consistent with this section.
================
sec 501..
(1) Before the 2013-14 school year, certificated instructional staff who have attained certification from the national board for professional teaching standards shall receive a bonus each year in which they maintain the certification. Beginning in the 2013-14 school year and each year thereafter, certificated instructional staff who have received a bonus for at least two years must be evaluated by the principal as a top tier teacher under RCW 28A.405.100 in order to receive an annual bonus. ....
Notice that trained observers put only 3% of teachers in the Top-Tier "Distinguished" in the two year Chicago study.
----
From Rethinking Teacher Evaluation in Chicago's
four levels of:
Distinguished
Proficient
Basic
Satisfactory
"Basic" teacher's evaluations tend to mirror the VAM test scores far more accurately than the "Distinguished" teachers.
Principal's evaluations were also far more consistent with trained observers observations.
This looks like the "Big Brother is watching you plan" to reduce spending on teachers rather than an effort to implement an evaluation system to promote improvement in each teacher's skills.
Check the graph on page 11 of the report. Notice this report is only for math and reading. Also note that the difference in actual VAM measures between an Observed rating of Basic and an Observed rating of proficient in math as close to zero.
Basic => -0.104
Proficient => -0.006
with "Distinguished at +0.522"
Given that Hattie found huge effect size differences in instructional materials and practices ... is it reasonable to apply such a standard to teachers in an environment that emphasizes Problem Based Learning 0.15 or Discovery Inquiry 0.31 rather Mastery Learning, Direct Instruction, Problem Solving instruction etc which have effect sizes about 0.60?
A comparison of Principal Ratings and Observer ratings shows the following for all the teachers rated:
...................... PR : OR
Distinguished . : 17% : 3%
Proficient ...... : 53% : 67%
Basic ............ : 27% : 28%
UN-Satisfactory : 3% : 2%
SB 5914 appears to be poorly thought out and essentially equivalent to buying a house that is still under construction and based on an imaginary non-existent blueprint.
===================
This is yet another example of why adoption of the Common Core State Standards and acceptance of all the Ed Reform pushed by RttT .... was and remains a really bad idea.
===================
WEA and SEA were supporters of CCSS in 2011 and of 6696 in 2010 which lead to all this mess and yet the members continue to let Union leadership sell them down the river.