Disqus

Saturday, November 12, 2011

This is a guest post from Jack Whelan who ran for School Board in District II in the primary.

“I have a button that reads: "Those who CAN teach. Those who CAN'T become education policymakers." Well, not all teachers can teach, and not all policymakers can't, but ... it's mostly true.” –Deborah Meier


It’s mostly true of school administrators, too. And that’s why it’s just wrong to have a teacher evaluation system that depends on one person—the principal. It only makes sense in a world where you accept that schools are like businesses where you get your performance evaluation from your boss. That’s frequently problematic even in corporate settings if you have a jerk for a boss (like that never happens) or a boss who, for whatever reason, doesn’t know your job because he or she never did it. (Ask any special ed teacher how many principals she’s had who have even the remotest clue about what she is doing.)

A boss who doesn't understand what you're doing can only evaluate you by some “objective” rubric/metric that measures productivity, but that can only work when your job involves doing something for which your productivity is objectively measurable. And so in order to make teacher productivity measurable, there’s this push to link teacher productivity to student test scores, and test scores then become inordinately important for all stakeholders, and getting them up becomes the really only important thing. And in order to get test scores up, you have administrators sitting on classroom teachers to make sure they are teaching to the test. This is inevitable once you accept the necessity of data-driven, quantitative productivity measurements.

But education just isn’t a quantitatively measurable, productivity-producing kind of thing—not at its core, anyway. The most important and memorable things that happen in a classroom are the un-measurables, the intangibles. It’s all about quality--the quality of the relationships, the quality of a teacher’s experience and judgment about how to meet kids where they are and to help them take the next step that’s right for them. (That’s quite a different enterprise from getting them to “standard”.) It’s about the quality of discovery, the quality developed in enabling kids to learn to love learning, to wake up to a world that is much bigger and richer, more interesting and more important than what can be measured on math and reading tests.

I’m sure that if you talked to most supporters of technocratic ed reform they would tell you that they agree with everything I’ve written in the previous paragraph. But then they’d ask, “What about the achievement gap? And then they'd remind me we're in crisis. And then they'd tell me we have to be aggressive and methodical in solving this world-historical problem. “We’ve got to get serious," they'd say. "We’ve got to knock some heads. We need systems; we need processes; we need ways to measure whether we are making progress. Yes, even a few would say, we need to impose the educational equivalent of martial law in our schools, and we need to give superintendents and principals the power to flush out or punish non-producers in the teaching ranks."

I don’t believe there’s some evil capitalist plot to take over education. Sure, there are opportunists on the periphery of technocratic reform, but that reform impulse gains the broad support it does because decent, intelligent people become too easily seduced by crisis thinking: we must do something, anything. But this compulsion to engineer top-down action plans that we impose on intractable, deeply rooted, historical and cultural problems almost always leads us to ruin. There were a lot of very smart, decent people who thought invading Iraq was the right thing to do. It just wasn’t, and it proved ruinous. The mentality that brought us Iraq and the mentality that is bringing us No Child Left Behind and Race to the Top ed reform are, I would argue, very similar. It's bi-partisan wrong-headedness.

Here’s what wiser heads understand: Bend your efforts to improve quality first, and the quantitative (standardized test scores, etc.) will take care of itself over time. The criterion to evaluate the effectiveness of a program ought not to be whether it has helped to close the achievement gap. Other fundamental structural economic and cultural issues need to be dealt with if we’re genuinely serious about closing achievement or opportunity gaps. Once we live in a truly equitable society, then we can expect scalable, equitable educational outcomes.

In the meanwhile, we focus on what’s best and what’s possible for each kid. That’s why I like Kate Martin’s ideas about mentors that would work with families and kids to define goals and shape strategies for what would be best for them, rather than have the district dictate what is best for them. Better the money go into that than into MAP testing, IMO. I hope Kate keeps pushing for that and that others support her.

We all want excellent schools, but excellence is relative to the particular challenges each school community and each kid in that community faces, it cannot be standardized. And you cannot force excellence. You inspire students’ aspiration toward it, and then find ways to cultivate it in them. And teacher-peers, parents, and—when they’re older—the students themselves are better judges of whether teachers provide that quality than the typical harried administrator--and they should be involved in a fairer, broader, relationships-centered teacher-evaluation process. That also was an idea Kate was pushing for, and the union should push for it, too.

Quality is something that develops and grows primarily in and through human relationships. We should be bending our efforts to promote high-quality, rich, collaborative relationships rather than obsessing about these quantitative measurements that are indifferent to quality and that force us to ask the wrong questions and worry about the wrong things.

52 comments:

peonypower said...

Thank you Jack.
You summarize so much about what teaching is and is not and how it can be evaluated.
India Carlson

dan dempsey said...

And so in order to make teacher productivity measurable, there’s this push to link teacher productivity to student test scores, and test scores then become inordinately important for all stakeholders, and getting them up becomes the really only important thing. And in order to get test scores up, you have administrators sitting on classroom teachers to make sure they are teaching to the test. This is inevitable once you accept the necessity of data-driven, quantitative productivity measurements.

I am in complete agreement with Jack. This may come as a shock to many of those who see me as the Test Score Guy. --- To improve a system requires the intelligent application of relevant data. -- W. Edwards Deming

"Intelligent Application" has hardly been a strong point of the SPS Central Office folks who spout the mantra of data-driven decision-making.

Five years ago I chose the data emphasis, as I knew no other way to resist the NONSENSE so rampant in decision-making.

What I have seen is decision-making that ignores the data to push the NONSENSE "all ahead full". Often we observe the violation of RCWs and WACs as well.

Most recently Board ignored the WAC requirement for a careful review of all options for closing achievement gaps before requesting TFA corps members "conditional certificates" under conditions warrant.

Now after the New Tech Network $800,000 purchase and the diversion of funds into Cleveland NTN STEM high school, I find that year to year HSPE test score comparisons for Cleveland are generally worse. Yet the District now seeks a waiver from the 150 hour requirement per credit.

A quick look through the revised Waiver Action report shows .... a complete failure to make the request in a manner that meets the requirements of the WACs.

The big crown jewel Cleveland STEM is not doing the job when Test scores are analyzed.... So what is up with that?

BIG SPENDING ON Technocratic solutions ... produced poor results .... but does that matter to the Board?

Clearly Action report authors Enfield and Tolley seem just fine with poor results or did they not notice results? (What is data-driven decision-making for them?)

[[ The TFA appeal is being handled by Freimund, Jackson, and Tardiff in Olympia .... The Board ignores the requirements of the WAC, board testimonies, and letters .... so now dumps dollars to Olympia lawyers]] --- I wonder if they will pay anymore attention to the Cleveland waiver request and how inadequate it remains even after revision on 11-2-2011.

Anonymous said...

Oh jeez. Cry me a river about the accountability measure by 1 person. The teacher's union balks at peer review too. That would give teachers more than data point, more than one person, to evaluate their work. But. No can do. Somebody might say something bad about the poor teacher. It could be unfair. It might be subjective. We can't have that. The union wants objective... no matter how ridiculous it winds up being. You can't then go whine about the principal.

-parent

Anonymous said...

The passing of Roscoe Bass is another reminder of how different most principals now are compared to the past.

Principals now teach for awhile and then go for the big bucks. Some were not even in teaching long enough to become good at it.
They simply do no know or recognize the nuances that occur in a classroom beyond the checklist of a "walk through."

There is a disconnect from the long term commitment to making students' lives better. Many principals, particularly in Seatte, are on the way up the bureaucratic ladder. Sadly, they often don't even know what good teaching involves, yet they are evaluating teachers.

--the system is rewarding the stone steppers

Numbers aren't the whole story said...

Dan, based on your own numbers in the next thread down, it looks like at Cleveland, black students have a better pass rate and lower "far below average" rate than at Ballard, Garfield, Hale, Rainier Beach and Sealth.

You say "What's up with that?" and that it's (Cleveland) "not doing the job" but it looks to me that it IS doing the job for more black students that at two of the district's "top" schools, Garfield and Ballard. The fact is that They aren't doing any better, in fact, they are WORSE for black students.

Yet you don't hammer them day after day after day. WHY IS THAT and what do you want to happen at Cleveland? Close it? Take STEM away? Have the district admit that it doesn't reach black students at ANY school at the rate it reaches Caucasian kids? You put the numbers up over and over and over and rap on Cleveland over and over and over but I never see you call out other schools for similar OR WORSE numbers. And I never see you offer anything but dropping Discovery math. It's a bigger problem than that and you know it.

And if you spent 5 minutes talking to parents who CHOSE Cleveland, you'd know that not all of them would want it shut down or STEM removed or whatever your secret plan is for that school. It offers a challenge in courses some parents actually want to see, even if YOUR enemy reformers put it in place.

The support it has speaks to why reform has traction in some communities because the "good schools" in "good" parts of town hardly do a "good" job for black students. You know that but it's more your style to just crap on STEM because it plays well to the anti-reform crowd.

I think you have some good points and you have for a long time. But you offer a skewed reality and ignore equally damning data to try and prove your points, whatever they may be.

Anonymous said...

"Achievement comes to denote the sort of thing that a well-planned machine can do better than a human being can, and the main effect of education, the achieving of a life of rich significance, drops by the wayside."

John Dewey, speaking from beyond the grave

mom of 4 in sps said...

so jack - you write very well, but what do i as a parent receive to know whether the teachers my children have are doing right by them? i cling to the map because it's one of the incredibly few objective, comparative data points i ever get, giving me a consistent year-over-year measure that also compares to same-grade peers.

i've met all kinds of teachers and principals in my kids' 33 collective years of school, and find it incredibly difficult to distinguish among the ones who talk a good game but don't have much behind that, the ones who may not be parent-friendly, widely-liked, or vocal but who may be just the educators my children (and others') need, and the ones I have no bead on whatsoever.

compared to my job, where i'm perhaps subject to the whims of a supervisor but otherwise have discrete objectives and goals i can and do answer to, teachers seem to operate relatively observation-free in their classrooms seem to have not so bad a deal (in the area of evaluation - not speaking to the overall difficulty of the job).

i don't seem to hear teachers offering up ideas as to how they might be measured - only that they should not be measured at all - or at least that's the meme that makes its way through the media and through localized sources such as this blog.

dan dempsey said...

Cleveland scores in first year of STEM dropped for Low Income and Black students on the HSPE.

Look at the only OSPI scores that are available from one year to the next during the STEM era. HSPE scores for Reading, Writing, and Science.

2010 before STEM
2011 year 1 of STEM

Low income students Difference from District
pass rates 2010 then 2011
-0.6% : -0.7% Reading
+2.0% : -2.6% Writing
+1.4% : -2.6% Science

Black students Difference from District
pass rates 2010 then 2011
+4.8% : -1.6% Reading
+8.9% : -11.0% Writing
+6.5% : -3.3% Science

Please notice that in 2010 the year before STEM ... CHS scores exceed district averages in 5 out of 6 categories above.

Scores for year one of CHS NTN STEM show 6 out of 6 below District averages.
=========

The CHS waiver request on the Action Report seeking Board approval on 11-16-2011 completely ignores the WAC requirement for a waiver.

dan dempsey said...

I thought I pointed out that 85 minute periods for 180 days at Cleveland have made a difference in Math ... but also remember that Cleveland students are receiving two credits for that CHS Algebra class not 1 credit like everywhere else in the District.

I believe that Algebra and Geometry are the only math classes that get the 180 day 85 minute per day treatment at Cleveland High School.

NLM said...

I agree with mom of 4. MAP may be imperfect but it provides a way to compare my child's progress over time with students locally and nationally. It's also used by schools in districts we've moved from and will move to so I know her new teachers will be able to use the results. As a family that moves a lot, sometimes mid-year, I cannot overstate the value of quickly getting a school/teacher up to speed re: what your child knows/can do. Having your child languish for several months every few years while folks 'get to know' him/her is just not cool.

dan dempsey said...

NUMBERS AREN'T THE WHOLE STORY wrote:
Yet you don't hammer them day after day after day. WHY IS THAT and what do you want to happen at Cleveland? Close it? Take STEM away? Have the district admit that it doesn't reach black students at ANY school at the rate it reaches Caucasian kids? You put the numbers up over and over and over and rap on Cleveland over and over and over but I never see you call out other schools for similar OR WORSE numbers.

==========
First ... I have frequently pointed out that the SPS does a horrible job of educating educationally disadvantaged students .. there are practices proven to work that the SPS ignores.

Second ... I have called out many of the "Better:" schools for lousy math performance in this District SEE HERE for 9th grade low income scores by high school.

Third ... the SPS continues to pursue practices that do not work. When Cleveland teaches algebra everyday for 85 minutes which is 30 minutes longer than 55 minutes per day (30/55 = 55% increase in time) and get results for Low Income students that are worse than Franklin and way worse than Clover Park high school. I hardly find this to be a validation that Cleveland's approach to math is meritorious or well thought out.

Fourth I think it would be wonderful if the SPS admitted they are doing a horrible job in teaching educationally disadvantaged learners in variety of areas and at every grade level.

What would be even better would be an admission from the District that they ignore the results from very large high quality research studies to continue inflicted this folly on students.

==============
In the next week I expect the District to either satisfy the WAC requirements for a credit waiver or have the Board reject the submission of the waiver as written.

If the Board chooses to follow unproductive plans and the public keeps reelecting the Board .... that is beyond the control of thoughtful persons.

If the Board chooses to neglect to follow RCWs and WACs thoughtful folks should hold the Board accountable with legal appeals. .... and perhaps recall proceedings.

Jack Whelan said...

When people with bad ideas are doggedly persistent in pushing them, it doesn't matter how vociferously you shout No, you keep getting pushed back if you don't have a Yes Message to push forward with. Because we have accepted the battle on terms offered by the reformers, we are continuously fighting on our heels.

This post is simply a feeble attempt to change the frame of the argument by saying that what matters is not measuring gaps, but delivering quality.

So, 'Mom of 4'--I hear you saying is that there were times during your kids' education in SPS when you wondered if they were getting the quality you think they deserved. and something like the MAP you would find reassuring in that respect. But my question to you, is that really good enough? You should demand more of your school than that, and you should work to have the schools develop more robust assessments that show genuine mastery, not just the kind of minimal snapshot info that such a test can give. Don't settle.

And it's utterly ridiculous to link teacher performance to it, don't you agree?

When my son was in SPS he had mostly competent teachers, some very good ones, and some not so great, but none who deserved to be fired. There are very few who deserve to be fired. So the real question is what do you do with the teachers who aren't that great? Is the principal the one in the best position to help that teacher to improve? Very rarely. So who is? Well, other teachers.

So "parent" you're right, the union is stuck on this rubric principal evaluation process for good and bad reasons, and it's up to teachers to stop settling and push their union reps to get something better.

But I suspect they won't push because good teachers don't care because they have always done well in their evaluations, and don't feel threatened. So there's a kind of complacency about the whole process, but the next thing you know, because the union keeps losing ground year after year, that even the good teachers will find they no longer measure up because what they do well and what administrators want are incommensurate. And then it will be too late.

dan dempsey said...

NLM said:
MAP may be imperfect but it provides a way to compare my child's progress over time with students locally and nationally.

It all depends on how imperfect the tool really is, as to how much worth it may have.

dan dempsey said...

NLM said:
But I suspect they won't push because good teachers don't care because they have always done well in their evaluations, and don't feel threatened.

It is now becoming very apparent nationally that there is a lot more pressure on teachers who are teaching in low performing schools....... As a result the more experienced teachers are moving out of low performing schools to higher performing schools.

Fine teachers in low performing schools who teach low performing students are often times the subject of persecution. ..... Well you are just not moving through the material fast enough, we expect some of these developmentally delayed students to move into regular gen ed classrooms at grade level next year.

Spin some straw into gold while you at are at it.

Jack Whelan said...

BTW, I am a Deming fan, Dan. And a Dewey fan, and a Deborah Meier fan, although her commitment to constructivist teaching techniques is too absolutist IMO. I believe in (and practice) a mix of both direct and constructivist instruction depending on the needs of one's students.

I'm not against standardized tests used as diagnostic tools. It's a question of cost and of trusting that they will be used for the limited purposes for which they were designed. In the long run, I would argue, if quality issues are the primary focus, there won't be any need for such superficial assessments.

NLM said...

Is the suggestion that an individual school-developed assessment preferable to an adaptable assessment that is nationally normed? I can't see how that would be any kind of improvement observe Hodge podge mix of state by state tests we got with nclb. The level of parental oversight being suggested presumed we dont have any other commitments. I don't want to babysit child's teacher. I just need an objective assessment of what's been learned.

jack Whelan said...

Let me put it this way. If you sent your kid to Lakeside, Bush, Northwest or Holy Names, would you be disappointed to learn they didn't have MAP testing? Maybe you would, but I don't think most people would. Why? Because they trust the brand. What's behind the brand? Quality--a culture and tradition of Quality.

Listen, the main difference in cost between public and private schools is the student teacher ratio--that's what all that tuition is buying--and the quality brand.

Are the teachers as individuals in those private schools so much superior to the teachers in our public schools? I don't think so, but the culture of the school sets expectations for quality performance from everyone--students and staff, and those expectations are not measured or calibrated by standardized tests. Why should it be any different in public schools?

When my son went through the IB program at Ingraham, I trusted the brand. I trusted the teachers and the curriculum. I knew that if he made it through that program, everything else would take care of itself. There are high standards in IB, but not standardization and bubble tests. That kind of thing would be silly and irrelevant. Why should it be different with any other program in SPS?

I think the problem lots of parents have with SPS is that they don't trust the SPS brand. That's a point Michelle Buetow made in her campaign. A lot of it has to do with lack of rigor in the curriculum. That's a point that Marty McClaren and Sharon Peaslee made.

So the question is how do you improve the SPS Brand? Do you impose some central corporate control that insures every school is the educational equivalent of an Applebees, or do you create the conditions in which different schools develop their own cultures and traditions of Quality? Applebees is fine, but I think it's "settling". I'd rather Wild Ginger or Canlis. I think Seattle has the resources to create such a public school system, but it needs the imagination and will to do it.

NLM said...

Jack - actually, yes, I would be dissapointed with a private school that did no nationally-normed testing whatsoever. I think every system needs some kind of external check to validate whether kids are being well-served.

My child was enrolled at one of best private schools in our old area and when we moved here I realized that despite the "brand" and the well-rounded education she received, there were definite gaps in her knowledge. Might those have been resolved the following year? Maybe. Then again, maybe not.

Don't get me wrong, my child was well-above average despite having fewer than 10 nights of homework the previous year but there were gaps and I didn't learn of them until after I got the MAP report.

Do I think SPS has a less than stellar brand? Absolutely. I was definitely more inclined to trust the old school because the class sizes were small (8:1 ratios in PK-2) and I believed they would know my child well enough to ID and correct knowledge/skill gaps. What I've come to believe though is that some of that trust was misplaced and that blindly trusting that you get what you pay for is also a mistake. Every child is different and minority children, even in affluent areas/top schools tend to be underserved/held to different standards.

I also don't expect that level of one-on-one attention to be free. It's not really necessary if DH's/my experiences are any indication. It's more of a personal preference, an extra layer of assurance/insurance that comes at a premium. My kids' schools were a)cost prohibitive for many b)difficult to scale up and c)entirely too autonomous. Public schools, as Seattle Citizen reminds, have to be accountable to every Tom, Dick and Jane with the right to vote. That does not lend itself to excellence. Too much compromise is required for governance.

Sure, I'd love to see more innovation, more choice and definitely higher standards but I don't expect the public to fund a private education for my child. I just don't think that's reasonable.

So, to answer your question...how to improve the SPS brand? I'd take a top down Ruth's Chris over a collection of freely operating Wild Gingers and Pancake Houses anyday. Within a highly structured system with high expectations, I think theres still room to add regional favorites/flavors to the menu.

Melissa Westbrook said...

"i don't seem to hear teachers offering up ideas as to how they might be measured - only that they should not be measured at all - or at least that's the meme that makes its way through the media and through localized sources such as this blog."

Okay, and once again, where on the blog has anyone said "teachers should not be measured" or "it's impossible to measure teacher performance". Because if it was said, I missed.

I've heard discussion of the difficulties, that one test should not be the whole for performance, that the principal should not be the sole person doing the assessment - but not that they can't be assessed.

NLM, I don't know of any private school that does nationally normed testing except for kids to get in and tests like the SAT/ACT. Anyone?

I think Jack is spot on about the branding of SPS. Many of us have been calling this out for years, I know parents in marketing who were willing to do pro bono work for the district and yet we have this yin and yang rep. But more than any single school or scores, it's the perception that this district is not well-managed. You just don't see Bellevue and Lake Washington with these kinds of scandals and constant missteps.

I re-read the Sutor Report recently and what depressing reading. It's such a snapshot of poor practices and for all the recs, you wonder, "Why wasn't this being done in the first place?"

NLM said...

The ntl. blue ribbon schools program honors both public and private schools based on test scores and so yes, private schools do test. Testing at the time of admission (or requiring parents to obtain test results) seems most common.

dan dempsey said...

Within a highly structured system with high expectations,

So where is this happening?
Did anyone think this was happening under MGJ?

---- Where is the actual accountability on those doing the structuring?

We have a system in which improvement is unlikely to occur with much speed .... because almost everyone involved is defensive....

The current focus is on Dumping Teachers through a bizarre formula.... it is hardly on improving instructional practices and student learning.... for if Improving the learning opportunities for each student was the goal ... then Hattie's Visible Learning would be used rather than ignored.
====

It was pretty hard to miss the nonsense when the Board was voting to spend $300,000+ on more Everyday Math supplemental instructional materials ..... while looking to evaluate teachers with Value Added tools.

Provide Crap materials and then use tools to hold teachers responsible "when the crap cannot be made to work".

=========
RIF a large number of Teachers in low performing schools ... also happened in Tacoma Middle Schools ... as part of a grant's requirements. ((This was a major component in the Tacoma Teachers' Strike))

Race to the TOP is not based on anything proven to work...... well except extortion is proven to work in getting states to fall in line.

Is this the desired Top Down highly structured system for America's schools?
========

Consider these results from Auburn that did not buy the EDM supplemental materials and allowed each school to figure out how they would meet Auburn's Power Standards in elementary schools ... vs. Seattle's highly structured system.

OSPI annual scores for grade 3,4,5
Auburn 24 Seattle ZERO

43% Low income in Seattle
52% Low income in Auburn

Anonymous said...

I don't know about all private schools, but The Northwest School is pretty anti-testing — just one of the reasons we are there. They don't do any standardized tests as part of school, and don't obsess over the PSAT, SAT, ACT.

Small class sizes and procedural policies create a very strong working relationships between the school and the families. There's plenty of opportunity to see how your child is doing, with detailed per-class assessments (based on homework, class participation, tests and projects) coming home 5X a year. The coarse curriculum for the quint is described, then individual student grades on the various elements (adding up to a total for the class), plus a brief paragraph personalized to the student. My guess is that most teachers have <50 students to assess, some only 30 or so, depending on how many courses/classes they teach; it varies.

This system would be pretty unwieldy in a large school, but it is IMHO, ideal. Considering the school also elicits comments from parents and students 5X a year, and the result is some good info for teacher evaluation as well.

The school truly instills a love of learning in most of the students. My daughter, whose social scene is not school centric, could not wait to get back into the classroom come September. Schools need to do that — create learners, not test takers.

SolvayGirl

Andrew Davidson said...

@ SolvayGirl,

"pretty unwieldy" would be a dramatic understatement for that kind of assessment plan for an SPS high school teacher.

A full-time teacher there has at least 125 students (and many quite a bit more), so there just aren't enough hours in the week.

I think that is exactly what you pay for in a private school and I am sure it is well worth it.

Michael H said...

@Jack:"We all want excellent schools, but excellence is relative to the particular challenges each school community and each kid in that community faces, it cannot be standardized. And you cannot force excellence. You inspire students’ aspiration toward it, and then find ways to cultivate it in them."

Really? So excellence is defined based on one's "community"? While I agree that excellence cannot be forced, I disagree that excellence cannot be standardized.

I am not an teacher, and do not work in the teaching profession. But the view that excellence cannot be standardized is not a real-world, post-school reality. It sounds as though you are saying that, for example, minority kids' community should have a different standard for excellence than a non-minority kids' community (inferring that those minority kids' standard of excellence should be LOWER than those of non-minority kids - e.g. the achievement gap). That would appear to be somewhat of a bigoted position - IF (?) that is what you meant.

Private schools, on an individual basis (not as part of a larger organization such as the SPS), set their own standards of excellence. That standard is part of their brand and is part of what keeps them in business. If parents don't like their standard because it is too low or too high, those parents will go elsewhere. For those in a larger organization, such as the Catholic schools (Seattle Prep, Blanchet, Holy Names, etc.), the controlling organization (the Archdiocese) sets the ONE standard that is followed. And minority students that attend those schools do quite well in school.

From what I've seen in the business world - part of that post-school reality - either the organization you work for or the industry set the standards for excellence. You either meet them, or you don't. Progress towards those standards is applauded, and exceeding those standards (which could become the new norm) is celebrated and rewarded.

I think it is vital to instill a culture and standard of excellence in our schools, and that standard should be one that ALL students (and teachers, administrators, etc.), regardless of race, strive to achieve. That is the reality students will face when they leave school, either for college or the workplace.

Jack Whelan said...

NLM--could you specify the nature of the gaps that the MAP identified.

Mike H--When it comes to kids who grow up in poverty, more important than high standards and expectations is to provide the wrap around services that will get them ready to learn, and that is not primarily a school function--it's a broader social responsibility.

I know all about the soft bigotry of low expectations, and I know there are some kids who grow up poor who are capable of overcoming the crushing limitations poverty imposes, but on any meaningful, scalable level, there will be just as much of an achievement gap ten years, twenty years, thirty years from now for kids who grow up in poverty if we keep thinking that what happens in the school building is the primary remedy. It just isn't.

We live in a society that is growing more inequitable with each passing year, so I don't have high hopes of poverty going away any time soon. I think that schools are being asked to take far too much of the burden for this deeply troubling, scandalous situation.

And once again, I think there's a difference between standards and standardization. The International Baccalaureate Program exemplifies that. Deborah Meier's alternative public schools in New York and Boston exemplify that, as do hundreds of alternative public schools and neighborhood public schools world wide.

They have unique learning cultures that have high expectations and high standards, robust assessments, and they graduate kids who are capable of taking on the challenges of adult life whether or not they become civil engineers, doctors, computer scientists, or construction workers, car mechanics, and small business owners.

And once again, I don't have a problem with standardized tests, like the end of course tests administered by the state, so long as they assess the actual curriculum that was taught. But more important than the test, IMO, is the curriculum, which should be challenging, interesting, and have its own robust assessments. It's critically important that kids show rich mastery of all subject areas, and bubble test assessments simply don't do that. They tell you something, but not about what's most important.

My worry about statewide or nationwide core curriculum standards is that they will be mediocre and promote pablum curricula to avoid being politically offensive to some constituencies, and then there's the problem with textbook economics of scale, etc.

MAPsucks said...

Despite submitting my written direction to NOT MAP test my child this fall, his middle school started to anyway (goes to show who listens to parent's wishes these days). I know they automatically went into MAP mode because: a) there were no MAP scores for him last year (there were MSPs however); and b) they didn't know what "track" to put him in. This is the second full week of school mind you, despite record of his previous teacher's suggestions for placement. It's just "easier" to consider him as a number on a page, rather than get to know him, see his work, use deliberative formative assessments (that's right, MAP is NOT a formative assessment.)

So now who is pushing "idiotproof" teacher assessments? Only those who want to sit at their computers downtown, look at their "dashboard" for overall widget status, and tell those teachers to move those sticky notes on those ridiculous "growth charts".

NLM said...

I think we agree more than we disagree. IB is a perfect example of top down, high expectations (the brand means something; there are quality controls in place). I share the concern about generic curriculum that's lowbrow but from what I've seen of common core, it's not that way at all, especially in math.

SPS may be top down, but I certainly don't think it has high expectations. When there was choice, it seemed a lot like good schools and boutique programs for families in the know and underwhelming schools for less discerning, discriminating or unlucky families. Now, it's just a ugly mess of income-based segregation.

The knowledge gaps I identified using MAP were primarily related to math. My child hadn't been exposed to enough practical applications for math, and did too little work with manipulatives. We supplemented, on grade, at home over the summer and saw 9% growth in her math MAP with increases in every single subarea.

Did I need MAP to tell me that she was stronger? No. It was enough for me that she was more confident. It did help us get her math placement changed though and that's made for a much happier kid.

mom of 4 in sps said...

Melissa said: Okay, and once again, where on the blog has anyone said "teachers should not be measured" or "it's impossible to measure teacher performance". Because if it was said, I missed.

I'm not sure who you're quoting re "it's impossible.." because I didn't say that.

Perhaps you could point me to a place in this blog where a teacher or someone affiliated with teachers suggests a way in which they could be measured or evaluated.

This blog and the majority of the commenters have been all about board accountability - but I don't recall much said about teacher accountability.

And Jack - I'm not settling for the MAP as you seem to suggest - I just said I appreciate having it (in addition to every other bit of information I can glean about what my children are doing and learning at school)

And I should be working for some more robust assessment? I can tell you I won't be doing that - aside from the fact that I don't have the expertise or the time to do that, it's teachers, the teachers union, principals, the principals association and district administration who should be doing that.

Angel said...

MAP is another costly smoke screen given to parents and outside observers as some kind of relevant measure. With a +/- 3 margin of error and an accepted +2 as a measure of progress, how does that make sense? Let alone that the MAP "consultants" readily admit that it can produce wild fluctuations in the scores of higher-end students, which were very evident. I am a former teacher, who left after 11 years of teaching middle school, because of a Principal who has never taught [telling us how to teach], the dreary, "unchallenging" boredom of Writers' & now Readers' workshops dominating/replacing the Language Arts curriculum, and the increased micro-managing of teachers by the District through their consultants and lackeys. Oh, and did I mention losing a LA/SS block (60 +/- students a day), and changing to a 7-period day seeing 160 students a day in core classes?

NLM said...

I found the crosswalk for math which indicates the types of skills/problems mastered given the various subscores and used that to see where my child's relative weaknesses were. Even though DD was already at the high end there was definite room for improvement and growth. I don't think a 15 pt RIT gain is insignificant at all. Every test has a margin of error but if a mere parent can use the data that way and achieve that gain, why can't a trained professional? So far this year, I have not seen/been given even one assessment of my DD in math or reading, formative or otherwise. I went online to see the fall MAP but with conferences not until next week, a third of the year's gone. Sure, I've seen spelling tests and free writing exercises, lots of phonics packets too, but that's it. What's a parent supposed to do with that? I'm not interested in teacher productivity so much as a fuller picture of what my child knows an can do and where she's struggling/needs my help. If I ask, I run the risk of being labeled an interfering PITA. I want to trust but is that wise given what I know of expectations for my child? What others deem adequate doesn't always meet my standards. Is that so bad? What's a realistic alternative measure for the success of my kid and her school?

mom of 4 in sps said...

NLM - I so agree with you on this and I feel your question has been asked by parents many times but never been answered in a clear way - on this blog or anywhere in the various venues where Seattle schools are discussed:

"What's a realistic alternative measure for the success of my kid and her school?"

Or even plural - "what are alternative measures...?"

How do I know my children and others' are getting what they need in the classroom?

It's not enough for those in schools to essentially say "just trust me" any more than teachers would take that from principals, school board directors, or district staff.

Jack Whelan said...

NLM--

As with most things in life it's often a matter of trade-offs. Let me ask you this. If you had a choice of sending your child to a public school that had a well-earned IB-type reputation for excellence at every grade level mainly because of the strength of its curriculum and the quality of its teachers, but did no MAP testing, would you choose another school with a mediocre reputation that had MAP testing?

I guess the question is which is centrally important and which is nice, but marginal. What should we be fighting for the first or the second?

mom of 4 in sps said...

Jack - how does a school receive a "well earned IB-type reputation for excellence", lacking a highly-standardized curriculum known globally for its rigor (like IB)?

In my experience, reputations are made from very nebulous sources, generally word of mouth based on anecdotal information and published test scores. They often lag reality by a stretch, maintaining a favorable reputation long after they should, and lacking a favorable reputation long after they should have earned it.

I want and should have both - a tried and true rigorous curriculum (I don't care about the reputation) and data points that I'm not expecting to be my sole source of information

NLM said...

"If you had a choice of sending your child to a public school that had a well-earned IB-type reputation for excellence at every grade level mainly because of the strength of its curriculum and the quality of its teachers, but did no MAP testing, would you choose another school with a mediocre reputation that had MAP testing?"

I'm honestly not sure. Now that I know a little more about MAP and how I can get/use more detailed data, I *like* that. The control freak in me says that even if it's a mediocre school, I can at least figure out what I need to do to make it work. I concede, however, that for most folks, having the school with the better reputation would be preferable.

Still, building a system that relies on the strength of a school's reputation to assuage accountability concerns is risky IMHO. I thought that our old school's reputation would guarantee an education in line with my expectations but it didn't, not completely anyway, and I've come to the conclusion that only I can guarantee a solid education for my child. To do that, I need good information nd at the K-8 level, there's not a history of IB-type exam results to show me that the material is being covered as each year. For me, an elementary parent, MAP serves as that EOC assessment.

NLM said...

WTA - Mom, well said. You read my mind.

hschinske said...

NLM wrote: Even though DD was already at the high end there was definite room for improvement and growth. I don't think a 15 pt RIT gain is insignificant at all.

My son's middle school MAP reading scores show things like this: starting out at a 96th percentile level, next time up 4 points to be at 98th percentile, next time down 6 points to be at 91st percentile level, next time up 10 points to be at 99th percentile level, next time down 16 points to be at 71st percentile level, and next time up 12 points to be at 95th percentile level.

Math is even weirder. From the first score at 91st, the second went up 15 to 98th, third up 9 to 99th, fourth down 12 to 95th, fifth same score again, 93rd, sixth up 7 to 96th.

Anyone really think these scores show "progress" in any meaningful sense? It's a dang game of Snakes and Ladders here.

Helen Schinske

Jack Whelan said...

I hope this discussion was clarifying. It was for me. A couple of thoughts to sum up.

Everybody wants Quality; the disagreement lies in how schools can consistently deliver it. I am sincerely convinced that the kind of reform mentality that lies behind NCLB and RttT is profoundly misguided, and I also understand why lots of well-intentioned, decent people support it or significant parts of its agenda.

I saw a debate/interview with Diane Ravitch and Steven Brill a couple of weeks ago, and I thought Ravitch lost in the exchange badly because she came across as a pedantic nay sayer while Brill came across as a pro-active problem solver. I think Brill is dead wrong, but it was clear that he has a powerful, positive narrative, and the only way to compete with it is to present an alternative narrative that is even more attractive. And I'm convinced that the argument for Quality has to be at the heart of it, because I sincerely believe that the obsession with control, centralization, and standardization produces mediocrity in the long run, regardless of what might have been intended by the reformers who push it.

As with most things human, there is very little black and white, unambiguously right or wrong about anyone's point of view. Life is messy that way. So when people claim some kind of superior legitimacy because their decisions are data-driven, I'm always suspicious. It comes down to Quality again--the quality of the data itself, and the quality of the interpretation of that data. There are lots of different ways to connect the dots, especially when data is cherry picked.

Data has its role to play, especially when it is trustworthy, high-quality data. But in the end it's more about the quality of the minds and souls of the people who are working to solve the problem; it's about good people using their best instincts, their best judgment, and their creative ingenuity to find solutions that will produce the highest level of Quality in our schools. The compulsion to standardize, IMO, works against finding the best, highest-quality solutions almost always, almost everywhere.

NLM said...

Helen, did you plot those scores? Drill down into the subscores? Check out the crosswalk to see whether a particular grade level topic hadn't been covered yet? Each test is a discrete event and there will be highs and lows (especially if one is relying on everyday math which does not uild skills in any cogherent fashion). I do believe, however, that over time one can run a line through the test results to see the overall trend, look at the topics covered in each sub area to see whether a child has mastered each of those skills and adjust accordingly.

Jack Whelan said...

To tighten my original post, I cut out several paragraphs on the them asking the wrong questions worrying about the wrong thing. My focus was on international test comparisons like the PISA.

Anthony Cody at Ed Week takes this theme in an interesting direction. Here's the link: http://blogs.edweek.org/teachers/living-in-dialogue/

Several people have pointed out, that if you factor out the 20% of American students living in poverty, American students rank at the top rather than at the middle. So here's a good example of how you interpret the data, and is the primary conclusion to be drawn from it that our schools are in crisis or that our society and the fundamental values organizing it are the the real roots of our crisis?

NLM said...

I am still trying to figure out how, in the absence of any standardization whatever, one is going to ensure quality. Branding only works because a standard is maintained and fidelity is enforced.

NLM said...

And why on earth would you remove low income students from a systemwide evaluation? Is that how it's done in western Europe?

Anonymous said...

NLM asks an excellent question: "And why on earth would you remove low income students from a systemwide evaluation? Is that how it's done in western Europe?"

According to the PISA Executive Summary, “Countries of similar prosperity can produce very different educational results. The best performing school systems manage to provide high-quality education to all students.” Looking at this gap, PISA notes that in the highest performing countries, 70 percent of its students from disadvantaged backgrounds score as well as their wealthier classmates. In the United States, only 30 percent of these disadvantaged students perform as well as students who are not disadvantaged. Here is the link to the 2009 Executive Summary: http://www.oecd.org/dataoecd/34/60/46619703.pdf


--Moose

dan dempsey said...

Numbers aren't the whole story wrote:

And if you spent 5 minutes talking to parents who CHOSE Cleveland, you'd know that not all of them would want it shut down or STEM removed or whatever your secret plan is for that school.

I have no secret plans.... unless using practices know to be efficient and effective is a secret plan.

Moose wrote:
Looking at this gap, PISA notes that in the highest performing countries, 70 percent of its students from disadvantaged backgrounds score as well as their wealthier classmates. In the United States, only 30 percent of these disadvantaged students perform as well as students who are not disadvantaged.

Cleveland hardly needs to undergo another big change... what is needed is a switch to using practices known to work... The selection of NTN was completely absurd. The data clearly showed that NTN schools were under-performers. They performed worse than nearby district schools that had a more disadvantaged population.

I spent a lot of time running the numbers for a significant number of NTN schools.

I suspected that NTN under-performance would be found after spending 60 seconds in Visible Learning.

When effect sizes from 800 meta-analyses of worldwide education innovations results show something does not work.... I will usually find it does not work in a specific case.

From Hattie's effect sizes =>


[effect sizes from "Visible Learning" by Hattie : the hinge effect value of 0.40 or greater indicates an intervention is likely to bring success]

Seattle's current math direction is centered on:
a. Inquiry based teaching (0.31)
b. Problem based learning (0.15)
c. Differentiated Instruction (no empirical evidence)

Consider the effective practices Seattle chooses not to use:
a. Project Follow Through's recommendation for Direct Instruction (0.59).
b. Problem Solving teaching (0.61),
c. Mastery Learning (0.58), and
d. Worked Examples (0.57).

==========
So we find after one year of Clevland NTN STEM, which requires Project Based Learning in every class that low income and black students were served worse than before NTN STEM ..... this is hardly a revelation it happens at almost every NTN school.

It was particularly evident in CA API scores at New Tech Sacramento ... the school that Sundquist, Martin-Morris, and Enfield visited thinking it was a STEM school ..... there were no NTN STEM schools in CA. .... the level of research that goes into an SPS action report is pathetic.

Charlie Mas said...

There can be little doubt that MAP, the ITBS, the MSP, and other standardized achievement tests have a value - a real value. They help us to compare student achievement, either individually or in groups to benchmarks and peers. Some of them could also be tools for identifying gaps in learning or instruction.

I think they are marvelous tools when used properly.

There is, however, a real tendency to over-use them or to use them for unsuitable purposes. Some of them, for example, may not be suitable for use when determining student placement. Most of them make a poor tool for assessing the effectiveness of a teacher or a school.

The data from these assessments should really only be used to prompt questions, not to provide answers.

For the student family member who wants to know if their child is learning what they need to learn, there is no substitute for a knowledge of the Standards and a review of the student's work. If, for example, the student is supposed to learn the slope/intercept equation for a line, then check to see if it appears in the child's math homework and schoolwork brought home. Ask the student what each element in the equation y=mx+b represents. If the student is supposed to be learning about allegory, then ask questions about the symbolic meaning of the characters in their school texts.

The answer to the question, "Is my child learning what he or she needs to learn?" will not be found in a standardized test score. But if the score is low, there is certainly cause to ask the question more sharply and urgently.

mom of 4 in sps said...

Jack - you hope this discussion was clarifying?

In order for it to have been clarifying for me, you would have had to answer the questions I asked you, and you didn't, therefore, it was in no way clarifying or really even constructive.

It's pretty easy to preach to the choir here on the blog - a lot harder to deal with people who aren't already on the same page.

It's actually an appropriate note on which to sign off this blog for the next 4 years.

Charlie Mas said...

Again, I absolutely respect the perspective and the questions raised by NLM and mom of 4 in sps regarding standardized tests. They are right to stand up for standardized testing. I should probably do it more often myself. That said, they step into the hazard that standardized tests present.

NLM asks: "I am still trying to figure out how, in the absence of any standardization whatever, one is going to ensure quality."

It has been my observation that "quality" has no single universal definition - certainly not one that can be measured by standardized tests. One family's idea of quality is another family's idea of frivolity. And the other family's idea of quality is a third family's idea of a dangerously narrow curriculum.

I prefer a loose definition that turns on how well students' individual needs are addressed. The course of study that meets the need of a student working below grade level is not going to meet the academic needs of a student working at or beyond grade level. For me, quality in a school pivots on how well the school - because it is often beyond the resources and authority of an individual teacher - can meet the various individual needs of individual students.

Just because a school is a good school for one type of student does not mean that it will be a good school for other types of students. And, vice versa. Just because a school is not a good school for one type of student does not mean that it will not be a good school for other types of students.

This idea of "good" school and "failing" school is a fallacy.

Is Eckstein a good school? It is by most standardized measures and by reputation. Is Aki Kurose a failing school? Again, by standardized measures and reputation it is. Some might conclude that student achievement is enhanced by something happening at Eckstein and is diminished by something happening at Aki Kurose. Is it really? If the two schools swapped students would they also swap test scores?

I don't think so. I don't think anyone else thinks so either. Neither of these schools is doing anything extraordinary that has much influence over student test scores. Neither school is much focused on those factors that significantly determine academic outcomes (student motivation, value for education, opportunity resources, mentorship, etc.).

Would an Aki Kurose student do better at Eckstein? Some of them would - those who are the type of student that Eckstein serves particularly well. The rest are probably served just as well at Aki Kurose. There is reason to believe that Aki Kurose would actually be a better school for some students now at Eckstein. Don't try to convince their families of that.

Is there any reason to believe that Eckstein does a better job of serving students working two grade levels below Standard than Aki Kurose does? Aki Kurose has a lot of resources focused on supporting those students. If Aki Kurose does a better job with these students does that make Aki Kurose a better school - for those students?

It's just like investments. It's not a question of whether the fund is a good investment - it's a question of whether the fund is a good investment for YOU.

NLM said...

Charlie - I see what you're saying but I fundamentally disagree on the value of that loose definition of quality. That kind of subjective, everything is in the eye of the beholder stuff typically means the usual suspects will be held to a different, lesser standard. In private schools with great reputations, in public schools with high standards, even when controlling for SES, minority kids get short shrift and I don't see a way to verify or address that without uniformly high standards and data over time.

dw said...

even when controlling for SES, minority kids get short shrift and I don't see a way to verify or address that without uniformly high standards and data over time.

I'm sympathetic to the reasons for worrying about this, and I'm actually a fan of some types of standards-based testing.

However, the danger with the words "uniformly high standards" is that there is no standard that is both high and achievable to all groups of students.

For example, with an adaptive instrument like MAP, you could set an arbitrary goal of 98% of the kids in every school across the entire district to exceed grade level standards by 10%. That would seem like an extremely ambitious goal, and in reality it would be unachievable in some buildings with lots of struggling students. But suppose for a moment that with enough superhuman effort it could be possible even in challenging areas.

Even still, those standards would not be "high" at all for many Spectrum kids, and they would be completely worthless for the vast majority of APP kids. Uniformly high standards, in the way most people interpret it, is not only not achievable, but not even desirable. What IS desirable is not letting kids slide through the cracks, holding some minimum baseline standards, while at the same time encouraging kids who are thriving to continue to do so. By meeting ALL kids at the forefront of their current achievement levels, and supplying extra support for kids who need it. This district, as in many cities, does a crappy job on both ends, both with struggling kids and with high achievers.

Jack Whelan said...

mom of 4--

Sorry, i took your question as more of a statement, and didn't know that you really expected a response.

Anyway, I think that my answer to you is implied in my other responses. I'm not arguing against the use of standardized tests. They have their place at end of year or from time to time. I think that a legitimate case can be made for using MAP as a diagnostic tool along the lines suggested by NLM-- as long as it's used just for that. (Do we have reason to believe that it might not be used just for that?) And I think that legitimate questions can be raised about cost both in money and in time spent.

So does the test deliver some value? I'm not denying that, but I think l have legitimate concerns about how it becomes more important than it ought to in the hands of bureaucrats.

My real problem is with standardization. Whose standards? Perhaps we assume that standardization is going to raise the bar, but does it really? I think there's plenty of evidence that it does just the opposite.

I don't want the test to become the tail wagging the dog, but anybody with a realistic understanding about how bureaucracies work is naive if they don't expect the test like MAP, or whatever's next, to become a tool for impose greater levels of central control. In general, I don't trust central offices to make the best decisions; it certainly did not in its fuzzy math adoptions.

And anyway, I'd argue that the standard is carried by the curriculum, not by the test. I think parents and teachers together are in a better position than central offices to choose their own standards and the curricula that will help their students attain them. So I want to see more decentralization of curriculum decisions because I think more often than not it will lead schools to set even higher standards than would be acceptable by the standardized standard. And that goes for schools like Aki Kurose as well.

For instance, I don't know if MAP tracks well with kids who are using Singapore or Saxon. Maybe it does, but I could see "the test", whether MAP or whatever is next, being used as a reason to deny further waivers because these or other more challenging curricula don't align with it well.

The test becomes the tail wagging the dog when it demands everybody use the Applebees curriculum when some schools would prefer the Wild Ginger one, but the test doesn't align with it--so sorry, no can do.

Charlie Mas said...

The measure of quality that I seek does not have to be completely subjective.

Imagine a report with these statistics:

% of students working below grade level who received an intervention.

% of students working beyond grade level who received additional challenge.

% of students with a Level 1 score on the MSP last year who got a Level 2, 3, or 4 score this year.

% of students with a Level 4 score on the MSP last year who got a Level 1, 2, or 3 score this year.

% of students with a higher level score on the MSP this year than last year.

% of students with a lower level score on the MSP this year than last year.

That goofy looking graph that I have been using as an avatar for most of the past year would show the data I want.

NLM said...

Charlie - didn't OSPI provide some of that data in the past? I remember charts that looked like that about 5-7 years ago.

Why wouldn't MAP work just fine with Saxon and Singapore? I used Primary Mathematics to supplement with my child this summer without any conflict whatsoever. If anything, it bridges the gab between Everyday Mathematics and Saxon, solidifying concepts that Everyday Math assumes kids will learn by osmosis and that Saxon drills to death.

SPS is also one of multiple districts that adminster MAP and they're not all using Everyday Mathematics. MAP is not supposed to align with any particular curriculum, it's supposed to align with state/core standards.

I think the placement tests used at the school-level are based on Everyday Math tho, which is a huge problem for kids new to the district who understand math in more conventional ways. EDM has its own language.

I'm not sure why a performance standard has to be Applebees level either. I'm less concerned about everyone meeting standards at the same time, as in NCLB, than having standards that actually mean something and getting each student as close as possible. Value added goals, not minimum performance, seem to address that concern.

Fundamentally, I also think there's a disconnect between the ideal of local control and the reality that we live in a highly mobile global society. Skills/abilities need to translate wherever you find yourself, not just locally. If my spouse is transferred to Maryland or Connecticut in a year, I want to know that my child can enroll in PS there and not have to be remediated regardless of Washington or Maryland or Connecticut curriculum choices.

Jack Whelan said...

NLM--I don't think a performance standard has to be Applebees level either-- but it probably will be. My argument is based on my understanding about how politically driven bureaucracies work.

I don't think we disagree in principle, but more on the level of what is the best way to produce the highest level of quality for the most students. Every arrangement has its tradeoffs, its strengths and weaknesses. I just don't trust bureaucracies, particularly in the political environment in which we are now situated, and particularly given the performance of this particular SPS bureaucracy over the last decade plus.