Sunday, October 02, 2011

Danny Westneat column: The Kids are Alright

With today's column, Wise Up, Seattle, to Turnaround in Your Schools, Danny Westneat showed us that he is unaware of the disconnect between the schools of Seattle Public Schools and the central administration of the district.

The schools, he points out, are actually doing pretty well, consistently beating the state average for pass rates on the state tests in nearly every subject and nearly every grade.

He uses this evidence to - erroneously - leap to the conclusion that the district is doing something right. He attributes the high pass rates to actions taken by the district. How odd.

What was that the district did that raised the pass rates? Was it increasing class size? overcrowding schools? cutting summer school? cutting elementary school counselors? pulling support for students with IEPs? or disrupting communities?

Maybe it was the way the district wasted tens of millions of dollars on closing and re-opening schools, spent $1.8 million for no purpose, spent $4 million to build a student data warehouse, spent $1,000,000 on laptops for STEM, spent $800,000 on NTN, spent $700,000 on the web site upgrade, spent untold millions on consultants, or spent 9% of the budget on the central office instead of the 6% that every other district spends.

There are two much more likely causes for the "turnaround" that Mr. Westneat notes.

1) Teachers. We keep hearing that having a good teacher is the largest school-based determinant of student success (as measured by standardized test scores). So how about we give the teachers the credit for the improvement?

2) With even a bigger influence than teachers, families are the primary determinant of student test scores, so wouldn't it be the families - and not anyone in the district - who should get the credit for this improvement?

34 comments:

Anonymous said...

Charlie, what do you suppose teachers and families did differently that led to the improvements Westneat refers to? What's your evidence that they are the more likely catalysts for the changes in test score achievement? And are they therefore to blame for previously lagging achievement?

Our district definitely makes mistakes, but I am persuaded that it has also made a much more conscious effort to support teaching and learning in the last several years, particularly in the most challenged schools. Maintaining a critical perspective on the district's work is important, but what's wrong with acknowledging that the work of the district can actually influence student achievement, for worse, and in some cases, for better?

--Wondering

KSG said...

Danny doesn't provide links to sources. Because the data I can find says something different.

Charlie Mas said...

Wondering,

You raise excellent questions. If the families are the primary determinant of student test scores, and they are, then what has changed about the families?

At the same time, what, if anything, have the teachers been doing differently and what, if anything, has the district leadership been doing differently?

Has there been a demographic shift in Seattle? I hear stories of gentrification but I don't know if there is data to support that belief. I do know that more families with young children are staying in the city and enrolling their children in the public schools than have done so in the past.

Here's my point - some sort of attribution analysis is called for before I or Danny Westneat, or anyone else claims to know or understand the cause of the change.

Mr. Westneat has proclaimed that the district is doing fine based solely on this data and completely without any analysis. I'm saying that's a faulty process for analysis and - likely - a faulty conclusion.

Decoder said...

Wodnering, this is how it works here: If something bad is happeing, it is the district's fault, but failing students are failing because they have uninterest/uninvolved/uneducated families. There are good teachers, many of them, but they can't help these kids. If you took the same teacers and put them in schools full of "good" students, those teachers would have much higher success rates. But the district itself would have done nothing good to help.

When good things happen, the district had nothing to do with it, in fact, anything good came about DESPITE the district. It's all the parents and families and the sainted, much experienced teachers. But if you took those sainted teachers and moved them to schools with "failing" students, their success rates would plummet and it would all be on the families and the district.

Once you learn this, you will be better able to navigate this blog and what posts mean. You have to read between the lines and get the biases of the writers to know what they're really saying.

So Danny is wrong because he isn't faulting the district for everything and isn't lauding the right people.

Anonymous said...

Charlie - you're so un american!

You really need to re educate yourself - when you think of bosses or you think of people in charge, you have to remember that they're above you because they're better than you!

if you were better, then you'd be a boss, and you're not a boss because you're not better.

ballmer, stritkus, ... all of them are better than you because they're in charge and you're not!

are you getting it, yet?

look, from now on, whenever you start to think of some boss person,
just repeat to yourself "how great thou art" and just click on the following link -

http://youtu.be/Nf0vJiyeLIo

PullingForCharlie

Michael H said...

In even simpler terms, the teachers work for the district (they are district employees). Thus the credit Danny gives goes to the district, whether you like it or not.

Anonymous said...

Do I think our kids are doing substantially better than a few years ago? Not really. Variations in testing and subsequent data are a more likely source of good news.

Has the dropout rate improved? No. Are more kids coming to college prepared, minus remedial classes, for higher learning? No.

A scholarly article making the rounds about reform "improvement" over 20 years in Chicago (including the self-aggrandizing Arne Duncan reign)shows the test data improvements to be largely false.

I'll find the article in my email then post.

Test Data Doubter

Anonymous said...

Here is the article from the New York Times: Assessment of Schools is Flawed, Report Says.

Can we get the University of Chicago to study Seattle?

Test Data Doubter

uxolo said...

Decoder, people on this blog do talk about economic variables.

We all know that wiith enough finances, a student goes for tutoring or works with the parent who is able to tutor at home. Teachers in schools that have that sort of support will be successful most easily. Effective instruction and administrators who monitor within class success or its absence can then provide interventions. Extremes of poor quality instruction and poor support at home make for the absence of learning.
Poor support at home is not unique to lower income households. Some families have no desire or ability to tutor their kids through school. And they shouldn't have to.

The more kids in a classroom who have that outside support (test prep included) the more likely that classroom is a place for success and it shows up on test scores.

I do not agree that the family is the most critical ingredient. Maybe more like the salt - kind of brings out the flavor. And the more often you taste the full flavor, the more you want to reach that achievement level.

emeraldkity said...

Some families have no desire or ability to tutor their kids through school. And they shouldn't have to.

Perhaps they shouldn't be expected to do the actual tutoring, but they should be involved enough to facilitate it instead of throwing up roadblocks when help is made available.

ben said...

It would be really interesting to do a demographic analysis. If the kindergarten capture rate has risen theoretically the district is gaining more of the kids who went to (expensive) private schools which should push scores up but I think is a more recent phenomena that wouldn't show up in this data.

Ben

Maureen said...

ben, the same thing could show up in Middle School though. I have heard in the past that drops in scores at MS can be attributed to who leaves SPS for private at that point. If more middle class kids are staying for MS the scores could increase with the change in population.

Charlie Mas said...

Here's the thing - Mr. Westneat attributes changes in the scores to actions taken by the district administration. That's not reasonable.

The primary determinants of student test scores are home-based influences. That would be the first place to look for the source of the change.

The most significant school-based determinant, as the district leadership keeps telling us, is the quality of instruction. That would be the second place to look for a cause to the change.

So it is perfectly absurd for Mr. Westneat to attribute the change in the scores to decisions or actions by the district leadership.

Wondering mentions "a much more conscious effort to support teaching and learning in the last several years", but i'm not seeing it. Is it the coaches? Is it the focus on fidelity of instruction? Is it the curricular alignment already showing results?

I see the district sending help to the schools but not sending help to the students. The students are getting their help despite the district leadership. That's what I see.

I want to know if you see something different or if you know of some action by the district leadership which can account for the change - given that strategic decisions by the district leadership are pretty far down the list of test score influences.

Is the district leadership doing things that actually improve the home support for students?

Is the district leadership doing things that actually improve the quality of instruction?

Correct me if I'm wrong.

seattle citizen said...

"I have heard in the past that drops in scores at MS can be attributed to who leaves SPS for private at that point."

I've always wondered if drops in MS might be partly attributable to the increasing awareness of MS students: They're are less apt (perhaps) to "do well in school" for its own sake, or at the behest of their elders; they are becoming more savvy of the repercussions of grade (or lack of them.) When they get into, say, 7th grade they are pretty aware that with easy grades and social promotion, they WILL get into HS at the end of 8th grade no matter what.
The 8th grade MSP, for instance, matters not a whit to a student not interested in the test for its own sake, or what it might tell them about themselves. Unlike the 10th grade HSPE, an 8th grader still "graduates."

I wonder if MAP scores can start to show us in more detail such trends. While it might not be an accurate tool in many ways, it should show us trends such as an overall rise (of all students) through, say, 6th grade, then a decline in 7 and 8.

THAT could attribubable to flight to private schools, too, but if one looked at students who stay and deleted those who weren't registered anymore, I bet one would see a precipitous drop into 7 and 8.

Why? Because students are promoted willy-nilly,

This is an acute problem, the more so when students who are expected to have a certain skill set are sent on to the next level in a given discipline. I know that many high school students, for instance, will fail LA9, yet be in LA10 the next year anyway: They'll make up the necessary LA9 (needed to graduate) with a pass packet. No help to them, or their teachers, as they struggle with higher-level concepts without the previous skill set.

Chris S. said...

I bet I know what Danny is doing. He's lumping everyone together and Voila! we're doing fine. Remember the problem is the GAP, which you only see if you look at groups relative to each other.

Anonymous said...

When will we stop confusing correlation with causation?

If improvements came from district actions, the evidence trail should be abundant and clear.

Finding the proximate cause is not so difficult that people have to resort to correlation-based political arguments that prove absolutely nothing.

If DW and the district want to tout their successes, show us the proof. That's all I'm asking for. Typically, however, far too many smart folks, like Joni Balter at the Times lazily spout the populist slogans like, "they must be doing something right, because enrollment is up!" Really, Joni? It couldn't be the baby booms going on in every neighborhood, could it? Duh!

There are a myriad of reasons, identifiable and provable, for why things occur. Why are so many satisfied with taking people with agendas at their word? Scary. WSDWG

Charlie Mas said...

So what did families do that caused test scores to rise relative to the state averages?

Seattle families got poorer, but we got poorer at a slower rate than families across the state did.

2007 FRE: 40.5% (SPS) vs 36.8% (WA)
2008 FRE: 40.5% (SPS) vs 37.9% (WA)
2009 FRE: 41.3% (SPS) vs 40.4% (WA)
2010 FRE: 42.4% (SPS) vs 42.3% (WA)
2011 FRE: 43.3% (SPS) vs 43.5% (WA)

We went from having a higher rate of poverty than the state to having a lower rate of poverty than the state.

FRE Difference, SPS vs Washington State:
2007: +3.7%
2008: +2.6%
2009: +0.9%
2010: +0.1%
2011: -0.2%

Given the high correlation/causation between poverty and standardized test results, this should be noted as a factor.

seattle citizen said...

That's a good point, Charlie, illustrated by some very scary figures.

43% of residents of Washington are on FRL?

Almost half the state is in need of help?

That's just sad. Tragic. Inexcusable.

Anonymous said...

What I got from Danny's article was less about the reasons for the improvement but instead points out that the Seattle community tends to be harsh on our education system when the data, although I think averages can be misleading, shows otherwise. For example, people keep saying the parents don't like this, don't like that; however, if you look at the results of the climate survey, it shows that families are satisfied with the quality of education (80+%) and family engagement (80+%). I'm not sure about the rate and demographics of the respondents; however, this would tell me that the Seattle community supports the district which partly explains why they keep approving levies.

A friend of Seattle

Charlie Mas said...

A friend of Seattle,

The distinction that Mr. Westneat missed - and I guess you missed as well - is that people are happy with their child's school and their child's teacher, but are not happy with the district.

Mr. Westneat is trying to transfer the satisfaction people feel for the schools to a satisfaction that he thinks people should feel for the District. He doesn't see them as separate parts of the system as many of us do.

I think that my kids have received a pretty good education from their schools. Does that mean that Peter Maier is doing a good job? I don't think so. I wouldn't make that leap, neither should Mr. Westneat, and neither should you.

Anonymous said...

@Charlie,

I note your differentiation. Taking the example of the results of the climate survey, if parents on average are 80+% satisfied, then doesn't this reflect on what the District is doing? BTW, I don't subscribe to the notion of Schools vs. District; it is one system. There is definitely room for improvement, but why not view it as one system. It seems very unproductive to be an agent of divisiveness instead of collaboration and unity.

I do not agree with your comment about taking a leap; the data shows it. Let's dissect the data with the spirit to better understand areas of needing improvement instead of being so negative that it is not constructive.

What is the better way to model to a child or student?

A friend of Seattle

Danny Westneat said...

"So it is perfectly absurd for Mr. Westneat to attribute the change in the scores to decisions or actions by the district leadership."
Did I do that? I don't see that in this article that I wrote.
The reason it's not there is that I don't know precise reasons why Seattle's test scores have begun to turn around. I have my hunches, as a Dad with kids in the schools, but I don't know the reasons so I decided not to speculate. Anyway, the rising scores are good news, no? Check out the gains in middle schools especially -- very good news.
-- Danny Westneat

Anonymous said...

@ Danny - I agree, it is good news and should be a cause of a celebration to rally the city! Managment 101. I hope the data guys will dissect the averages and identify the areas that needs improvement and sit down with the academia folks to inform their work.

A friend of Seattle

Anonymous said...

"...of Seattle" at 4:42

The Noble Seattle Censor Attack -

my style of conversation is positive and constructive, and my thoughts are positive and constructive

and since you don't adhere to my style of conversation and to my thoughts,

you're neither positive nor constructive!

You're trying to model censorship through double think?

NobleSniffer

Anonymous said...

@Noblesniffer - Noted! But is the gang up approach any better? I was having a conversation with Charlie and I've read him over the months and have seen him speak in public and he has no issue expressing his views. ;)

A friend of Seattle

Anonymous said...

Right on Decoder. Danny didn't laud the saint parents, mostly who author this blog. It's always parents who are happy with their particular school that laud the great teachers. Then they laud other great schools, they too must have great teachers or great programs or great something. Then they dis the district because it must have done something wrong for someone else, and they were disrespected them in some way. Nobody listened to them on how to "help" other peopel. How many times have we heard there's something great happening at "Maple" or "Mercer"? It should be "replicated". How would these people know that? They also say oh well, the poor ye shall always have, and schools can't do anything about achievement bungles anyway. Then there's the measure of achievement. They all, of course, hate test scores, but think nothing of using them in analysis. What else is there? Oh yeah, nothing. How confusing. If it's all about the socio-economics and the family why would we do try to replicate anything? These people only have hearsay experience with these schools, and they aren't professionals. Not much different from their complaints with Westneat.

I took Westneat at face value. People love to loathe their school. I agree with that. My experience in secondary school, however, doesn't match his. There is plenty to loathe in my experience.

-Decoding with you

dan dempsey said...

Hey Danny Westneat,

Thanks for joining the conversation here.

The Middle schools have definitely improved on OSPI annual testing over the last few years.... and achievement gaps have been closing in middle schools.

Here is a NY Times article that addresses whether Chicago Schools have improved. It bears reading as Seattle may be somewhat similar to what is being reported by the University of Chicago analysts.

dan dempsey said...

If we wish to improve the teachers in our schools, perhaps a grow your own plan is in order.

Consider how a Black High School Student with an A average and 1320 SAT scores wound up teaching rather than in investment banking.....

John W. Williams III, 36 years-old, is a product of the North Carolina Teaching Fellows Program.

North Carolina is a State Grooming Its Best Students to Be Good Teachers...... meanwhile WA State undercuts teachers with Teach for America.

Go Figure.

It seems that Mr. William's kids are OK.

Charlie Mas said...

Danny Westneat wrote:

"'So it is perfectly absurd for Mr. Westneat to attribute the change in the scores to decisions or actions by the district leadership.'
Did I do that? I don't see that in this article that I wrote.
"

Oh! Let me point it out for you. There is always a potential for confusion - intentional or unintentional - when talking about Seattle's public schools vs talking about Seattle Public Schools. One, Seattle's public schools, are over 90 institutions scattered across the city, while the other, Seattle Public Schools, is one institution at the corner of 3rd Avenue South and South Lander in SoDo.

Mr. Westneat objects to the negative talk in the campaigns:

"You can hear it in the campaigns for Seattle School Board, and for the $232 million Families and Education Levy. Vote for me or for this levy, the theme goes, because our schools stink. They desperately need help."

Only the folks who are making this negative talk are talking about the district - the district leadership in particular - and not the schools. Mr. Westneat is - either intentionally or unintentionally - misreporting the negative talk. People are NOT saying "our schools stink"; people are saying our district leadership stinks.

Either Mr. Westneat has been confused or he is trying to confuse others. Either way he should stop.

Talking about Kate Martin at the Stranger candidate debate, he reported that "She had ticked off a slew of reasons the schools are broken and need a total rehab." but was Ms. Martin talking about the schools or the District? I think she was talking about the District - the District leadership in particular.

Mr. Westneat made a similar sleight of hand when quoting the mayor.

"'I don't think things are working right now,' he said, when asked why city government was getting more involved in schools. 'And I think the data shows it's not working.'" But the city government isn't getting more involved in schools, not all 90+ of them. The city government is getting more involved in the district - in the management of the district in particular.

Mr. Westneat's column is a response to criticism about the district leadership and his response is that the schools are doing well. That's not in dispute. The schools are, for the most part, doing very well. But it's the district leadership that was criticized - not the schools - and the district leadership is performing poorly. They do not deserve Mr. Westneat's defense and they do not deserve credit for how well things are going at the schools where they are not regarded as a positive influence.

Anonymous said...

If "the schools are doing well", the we could ask the same question of Charlie: How do you know the leadership is NOT doing well? Isn't the result of "doing well" an outcome of leadership? You may not like what they've done, or what how they did it, or how they listened, or how they XYZ... but if you like the the pudding, then credit the proof.

If you thought schools were "failing", then that would be another matter.

-observer

Charlie Mas said...

Thank you, observer, for your question.

The schools are doing well. Does that mean that the HR department functions well - or is there a set of performance measures for the HR department which are independent of student pass rates on state tests? I'm thinking that the HR department has other performance measures that are more specific to their work.

In a similar vein, don't all of the other headquarters departments (procurement, nutrition services, facilities, finance, accounting, legal, etc.) have their own performance measures which are completely independent of student pass rates on standardized tests?

So how would the student pass rates provide any evidence that speaks to how well any of these departments are performing? Not at all.

What about the Teaching and Learning departments - the various academic departments like Advanced Learning, bilingual education, Native American education, Special Education, mathematics, literacy, social studies, international education, physical education, art, and science? How can we measure the performance of these departments? Shall we measure them based on student pass rates relative to the state average or could we not find other, more specific, measures of their performance that are more directly linked to the work that these people do? I'm pretty sure that we can find other measures of their performance which are more closely linked to the work that they do with their own hands.

"How do you know the leadership is NOT doing well?"

We know that the leadership are not doing well because they got the worst audit that any school district in the state ever got. We know that the leadership are not doing well because they are having to re-open schools so soon after closing them, because they are cutting services to students while investing millions in pet projects, because they are expanding the executive administration while cutting librarians, because they are spending money foolishly, because they cannot provide data to support their decisions, because they fail to keep their commitments, and because they fail to manage the district. We don't know how well or poorly these headquarters department function because the district leadership has refused to perform any oversight.

So, no, the result of schools "doing well" is not an outcome of leadership. It is an outcome of the work done by the front line staff in the schools - the teachers, the students, and the families - often despite the leadership.

Ask the teachers, the students and the families if they feel well-supported by the district leadership. I'm pretty sure that they do not. I don't hear that sentiment voiced very often and I am actively listening for it. Wouldn't that be one of the best measures of the performance of the district leadership?

Each person working in Seattle Public Schools has their own performance measures. Each department has their own performance measures. For none of them - not even the superintendent - is the measure of their performance the student pass rate on state tests relative to the state average. It's no way to measure the performance of anyone in the district.

It's a good question observer and I'm grateful for the opportunity to address it. Thank you.

Anonymous said...
This comment has been removed by a blog administrator.
Charlie Mas said...

Another point - and I know that this is strictly for the academia.

We should not be measuring our peformance relative to anyone else. We should be measuring our performance relative to an objective benchmark. It is not a mark of success to be the richest beggar in the homeless camp.

Anonymous said...

Now there is an assessment I can agree with. You are right. No, we don't want a measure against state averages. That doesn't make sense to me. But I disagree with all the other logic.

Yes, lots of people complain about their boss. When do you EVER hear about "the great management" allowing the line to do the work? You never hear that. It is unreasonable to expect it. I can't think of a single enterprise where the line values the management, and the claim is something other than "those bloated managers".

Opening and closing schools isn't a measure of failure either. Times are different, economics have changed, demographics change. While it isn't great that open-close phenomena happened, it also isn't a clear measure of anything. A shfit to neighborhood schools is a good thing, even if there are growing pains. And as to audits, you can find an audit to support or decry anything. And they have. Do we really want more audits? That's an expensive fishing trip too.

If the net result of the line is satisfactory, then the management is part of that result. And, I'm not saying the line is "doing great" or even "satisfactory", just talking about the method of appraisal.

-observer