Disqus

Tuesday, August 18, 2009

Interest in WASL analysis?

I have seen some analysis of WASL pass rates - White vs. African-American, trends over time, cohort following - either in the District or school-by-school. These analyses were done either by me or by other blog participants; I'm not seeing any done by the District.

Is there some analysis that folks would like to see? Honestly, these things don't take much time and I'm happy to do them. I'm as interested in the results as anyone else, so don't be shy about asking for something.

Data is available for all schools in the state, so you could see how many high schools with at least 40 students tested and without the word "Alternative" in their name had lower pass rates on the math WASL than Rainier Beach High School (there were only nine).

10 comments:

jd said...

I'd like to see an analysis that follows a cohort. The 3rd graders of 2006 are the 4th graders of 2007 are the 5th graders of 2008. Are the same group of kids gaining ground or falling behind? This analysis would presumedly track the impact of the school better than a simple school-to-school comparison of a single year, for which variations are probably most correlated with demographics. (This wouldn't work perfectly however, since many kids in poverty transfer schools frequently -- however, it would still be an improvement).

I'd also like to see a graph of WASL scores vs FRL percentage. You could then see which schools fell above or below the mean line -- i.e. which schools are doing better than average, for their given demographic. It would also be interesting to see if the mean trend was a straight line or a curve -- If it's curved, you could imagine that there might be a "tipping point", where if you get higher than a certain percentage of FRL in a school, then the majority of the students are facing such huge hurdles that the whole classroom starts to struggle.

seattle citizen said...

Charlie,
As we (and policy-makers) go happily about (ha!) analyzing correlations, cause-and-effect, cost-benefit, program building/program ending...based on statistical analysis, I personally would like to see some mixing up of the variables, some additions to the "categories," and some qualitative data thrown in for good measure. Over the years, one can't help but notice the disconnect between assigning labels to individuals based on supposed group behaviors, trends etc, and the incontrovertible fact that individuals are widely disparate and the "products" of 7,12, 17 years of "factors."

I guess what I'm asking is for tools to look more carefully at the big picture, rather than thin little slices of supposed group identification.

This is the United States, whose crowning achievement (I think) is that it strives for diversity in all its forms: Live and let live, experiment, combine new ideas/cultures/peoples into a million different permutations (both economic and social)...But we seem insistent that it's possible to pidgeon-hole someone based merely on race, or class, or gender....

Ideal world: Teams of expensive analysts look at reams of "data," both quantitative and qualitative, and come up with detailed pictures of the many factors that help (or hinder) various parts of the whole child.

Instead of expensive teams, perhaps YOU'D do this for us? You seem to have some time on your hands! ("Please! Give me some data-crunching to do!" cries Mr. Mas...)

(I can't believe the Stranger actually criticized you for being intersted and willing to delve deeply into these oh sao important details of the education of our children. You rock.)

Anonymous said...

Actually, what I would like to see, Charlie, are yours, or anyone else's ideas for IMPROVING the gap in scores. There's a lot of talk on this blog about numbers and how dismal they are, but not a lot about how to change them.

You've mentioned in the past closing both RB and Aki Kurose, in part, because of the bad WASL numbers. I would like to know how that changes the fundamental problems that actually exist across the district-that overall, minority populations (black, Hispanic, Native) are lower-scoring and do not "achieve" in the numbers that that their white counterparts do.

I first became aware of the achievement gap well before I even had a child in this district-seems to me very little has changed over the last 10 years, other than white families fleeing the district and bloggers wringing their hands over those poor kids being so underserved at their terrible schools.

In my opinion, the problems and causes are many and often intertwined. Some teachers think black kids cannot learn or don't want to (see the school page on Examiniers.com for a post on this). Some black parents are young and undereducated themselves, some kids come from homes where parents are just trying to stay one step ahead of being homeless, some come from abusive homes, and some do just fine, yet hear enough about the lack of opportunity that they give up. And more.

So I am not really interested in how terrible RB did on the WASL last year. I am interested in how we go about getting entire populations of students better educated. Period.

jd said...

I agree completely abigean, but the data can actually tell you how to identify schools that are actually walking the walk, in practice, with real kids. We can then ask the staff of those schools what they're doing, and how they think their processes can be replicated in other schools that are having more trouble. While good ideas can of course come from outside the classroom/school, it seems worthwhile to me to start by identifying what seems to be working in an actual school. Analyzing the data helps to tell us this.

Of course, one can't stop there.

Charlie Mas said...

seattle citizen, I can analyze some things, but there are a lot of questions, mostly the Big Questions, for which I lack the necessary data to reach any conclusions.

Can do:

Correlation analysis such as the one jd asked for. I have done some really lovely and illustrative things like this, including the identification of outliers.

Tables and charts tracking cohorts.

Distribution analyses. I once did a really good one showing the bi-modal distribution of concentrations of poverty in neighborhood elementary schools. It was Killer.

I can also do, as seattle citizen mentioned, distribution analyses within categories. This is the sort of thing that Stephen Jay Gould did when he wrote his devastating critique of The Bell Curve in which he showed - handily - that the difference in average IQs among racial groups was inconsequential when compared to the diversity within the groups. Averages are usually such sloppy statistics that they are essentially meaningless. Think of all the yahoos who try to suggest that the average spending per student in Seattle Public Schools is enough to send them all to private school instead. It neglects to consider the fact that there are some students with disabilities who cost a great deal to educate. The cost of an average students is nothing like the cost of a typical student.

Can't do:

Attribution analysis. I can't tell, from the data, what caused the numbers to change. If there is an event - such as the adoption of a new pedagogy - I can indicate that qualitative data in the results.

Apply metrics for which no data is available. I have seen studies that indicate that children's academic achievement is highly correlated with the amount of praise they get. I, however, do not have data on praise. Also, it is unclear if this is a causative relationship or a correlated one.

Charlie Mas said...

agibean, how about this? How about we identify the schools that have been able to close the gap. Then, we can go to those schools and ask them what they did. Then, we can duplicate that practice in other schools. Does that sound like a plan?

I hope so. If you're onboard with that plan, let me tell you where we are with it.

We have identified schools that have exceeded expectations, Maple is most often mentioned, but there are others.

We have gone to those schools and asked them how they achieved that success.

They have told us.

Here's what we have not done: we have yet to make any legitimate effort to duplicate that practice in other schools.

So why is that? Why is it that the District talks ALL THE TIME about duplicating best practices and duplicating successful programs, but we have never (What never? Well, hardly ever) seen them do it?

The answer, of course, is site-based decision-making. We don't have a TOPS II because no school has decided to become TOPS II. We only have three language immersion schools because only three schools have chosen to become language immersion schools. There are 11 ALOs north of downtown and only 2 south of downtown because the schools north of downtown have started them and the schools south of downtown have not.

Only recently have we started to hear talk - no action yet - about the District telling a school to start a program. The District told Lowell to have an ALO and it will soon tell Mercer to offer language immersion.

Here's the big secret:

Establish and maintain high academic expectations for all students and support students who are having trouble meeting those expectations.

That's it. That's what the successful schools do. That's how they close the academic achievement gap by working to bring all students up to the Standards. They don't just talk about it; they actually do the work.

Here's something that is no secret: it is a lot of work. More than that, it requires sacrifice and prioritization. Something else almost certainly will suffer as a result.

That is the commitment that the District has made to the students and families and the community. That is the commitment that the District must keep. In the end, it is up to the Board to demand accountability on this commitment. Up to now, the Board has not done it.

Charlie Mas said...

What then, is the purpose of pointing out how horribly Rainier Beach High School did on the WASL?

The very purpose of the WASL was to measure the effectiveness of schools and districts, not the academic achievement of individual students. The test was supposed to be about holding adults accountable, not children.

So we should use the data as it was intended. We should use the data to confront the administration and staff of Rainier Beach High School and to demand better performance of them. If they cannot perform better, then we need to find other people who will. Or we need to get them the tools that they need. It may require very small class sizes. If so, then that's what we do. It may require an extended day. It may require an extended week. It may require a different pedagogy or materials. It sure as hell requires something other than anything they have done to date.

Unknown said...

This is a model I would appreciate being followed in Seattle: http://www.harlemvillageacademies.org/. (Note - these are charter schools - but the theme is fantastic: high expectations and discipline.)

Here is a great quote from the Harlem Village website: "Harlem Village Academies has high expectations for character, behavior and habits of scholarship. Every student is expected to arrive on time every day, wear a school uniform, and follow a strict code of conduct. We make clear to our students from the beginning that we do not tolerate late homework, disrespectful words or even gestures, or any other "minor" infractions. High expectations convey our respect for our students, and our belief in their tremendous potential. This engender in students strong self-esteem and respect for self and peers."

Children are *expected* to adhere to the rules and do well. No exceptions. School personnel consistently enforce this message.

There are parents of Seattle Public School students who display a complete and utter lack of interest in their child's education. That is an issue addressable by school employees - indicating that they care about the child's education and acting on this. Building a team that works towards that goal is very doable - Seattle has loads of stellar teachers and, in general, good team members want to be on a team with other good team members.

Given that, I've watched multiple PTA members (with help for school staff during normal school hours) wade into the land of social services *using school property and/or materials* and I think that needs to be off the table. Connect a student with help, but keep education and achievement the focus in the school.

seattle citizen said...

schmich,
You write regarding the Harlem Academy that "Children are *expected* to adhere to the rules and do well. No exceptions. School personnel consistently enforce this message."

Nothing new, there. Lots of schools make this claim: they "expect" that students will follow the rules and work hard. Regaridng enforcement, could you tell us how they do this? My understanding is that the Harlem Academy model requires a large commitment from the community to back up the school: parents, social networks, all in place and active in working to back up discipline actions and model and enforce discipline outside the school.
Which leads us to your next paragraph: "There are parents of Seattle Public School students who display a complete and utter lack of interest in their child's education. That is an issue addressable by school employees - indicating that they care about the child's education and acting on this."
So how DO we get whole communities wrapped around schools? How DO we get the "parents who display a complete lack of interest in the nrheir child'r education" to step up? Becuase you follow that sentence by saying that this current lack of parental support is addressable by school employees: Do you mean that school employees should bring the parents into the fold, or that school employees should make up for, in the classroom, the parents' apathy? Because THEN you go on to decry the "social work" done in schools, in school space and with the aide and abetting school employees who should, evidently, be focusing on education....

I'm just usure what exactly you propose.

Education necessitates "social services." Students are social, in all that entails, and this aspect of their lives often requires some sort of action or recognition on the part of educators (and the PTA!) Wouldn't it be nice if every student came to school rested, well-fed, nurtured, enriched, safe, warm....but they don't.

How do we get the oommunity to step up and model discipline, care, nutrition...for if they don't, damn straight it's left to the educators, and they'll do it if they have to.

Incidentally, could you steer us to data that demonstrates the tools they use to discipline students, to deal with students who don't follow the rules and do well? I've been to various websites, and I found one discipline policy I believe was Harlem's, and it didn't appear that much different than the SPS policy.

My bet is that the community has stepped up (and perhaps backed up the school if the school had to expel a student or two. The school reports 100% success on some tests, and if the discipline is so strict, one wonders where they are stashing their "trouble" students. Maybe I'm cynical, but it would take awhhile to convince me that ALL students, no matter who, is miraculously brought up to level in two or three years at the HA. Their discipline policy might be "toe the line or your expelled." SPS has no such luxury: A school can expel a student, but as the law says students have a right to school, SPS is repsonsible for teaching them somewhere. The cynic in me says that HA shovels the trouble out the door, but I'm hopeful someone will prove me wrong.

Dorothy Neville said...

I've been following this science teacher's blog for a couple years now. She sounds like a passionate caring and thoughtful teacher.

Here she analyzes WASL scores for a school where she taught for a year.

She's moved on from teaching into science education policy at the state level, although she wore several hats and got RIFed from one, so I do not know exactly what she is doing now. I don't know what district this is, not Seattle.