Performance Management
The District has released a number of seriously overdue documents relating to Performance Management.
The first is the District Scorecard. There are a number of interesting points about this document. First, there is absolutely no reason in the world that the District could not have delivered this scorecard on time in December 2008. There's nothing here that reflects 18 months of design work. Second, the scorecard, while it does show the results for each year does not show the annual benchmarks. Are there no annual benchmarks? Moreover, it doesn't show if the District had growth or was on pace in previous years. That could have been done with some simple color-coding. I don't understand the reason that the AYP data box in the lower right was included. Do we care about that? Does that tell us anything that we want to know? I would much rather that they used the space to show how many students scored 1, 2, 3, or 4 on the WASL. Pass/fail isn't as meaningful.
The School Report Card, which is also dishearteningly late, is still incomplete. That's just shameful. I was curious by what they mean by "Students making gains on the state math test." Do they mean this year's fourth graders over last year's fourth graders? Do they mean students who got a level 1 last year and got a level 2 this year? Is it a net measurement; did they deduct for students who suffered losses? What the hell does this mean? I read the explanation, and I still don't know what the hell the number means. They still don't have measures for engagement? They still don't have measures for leadership instruction or environment? What the hell are these people working on? After nearly two years they still haven't got anything here! I'm happy to report that the average class size is based on class size as of the October 1 enrollment for the respective school year, for all scheduled homerooms and classrooms, excluding special ed and elementary BOCs.
On the Middle School Report Card I wondered how they would determine if 8th graders are leaving middle school ready for high school math? What's the measure for that? According the explanation, it is "8th graders earning a C or better in an on-track or advanced math course. Advanced math courses include any Integrated Math course, Selected Math Topics, Mesa Math 8, Mathematics 8H, Mathematics 8, Middle School Math, and Mathematics 8E." Wow. The people who wrote this thing don't know that integrated math has been replaced. And what are all of those other courses? Hasn't middle school math been aligned?
The first is the District Scorecard. There are a number of interesting points about this document. First, there is absolutely no reason in the world that the District could not have delivered this scorecard on time in December 2008. There's nothing here that reflects 18 months of design work. Second, the scorecard, while it does show the results for each year does not show the annual benchmarks. Are there no annual benchmarks? Moreover, it doesn't show if the District had growth or was on pace in previous years. That could have been done with some simple color-coding. I don't understand the reason that the AYP data box in the lower right was included. Do we care about that? Does that tell us anything that we want to know? I would much rather that they used the space to show how many students scored 1, 2, 3, or 4 on the WASL. Pass/fail isn't as meaningful.
The School Report Card, which is also dishearteningly late, is still incomplete. That's just shameful. I was curious by what they mean by "Students making gains on the state math test." Do they mean this year's fourth graders over last year's fourth graders? Do they mean students who got a level 1 last year and got a level 2 this year? Is it a net measurement; did they deduct for students who suffered losses? What the hell does this mean? I read the explanation, and I still don't know what the hell the number means. They still don't have measures for engagement? They still don't have measures for leadership instruction or environment? What the hell are these people working on? After nearly two years they still haven't got anything here! I'm happy to report that the average class size is based on class size as of the October 1 enrollment for the respective school year, for all scheduled homerooms and classrooms, excluding special ed and elementary BOCs.
On the Middle School Report Card I wondered how they would determine if 8th graders are leaving middle school ready for high school math? What's the measure for that? According the explanation, it is "8th graders earning a C or better in an on-track or advanced math course. Advanced math courses include any Integrated Math course, Selected Math Topics, Mesa Math 8, Mathematics 8H, Mathematics 8, Middle School Math, and Mathematics 8E." Wow. The people who wrote this thing don't know that integrated math has been replaced. And what are all of those other courses? Hasn't middle school math been aligned?
Comments
You write:
"I was curious by what they mean by "Students making gains on the state math test." Do they mean this year's fourth graders over last year's fourth graders? Do they mean students who got a level 1 last year and got a level 2 this year?"
My guess is that it is the first: this year's 4s over last year's 4s. This seems to be how schools are "rated" in most of the "performance" indices I've seen.
Which of course is meaningless because the populations thereby compared are different...
Go figure.
RBHS does need new text books, not just for AP. Here she goes again, you say. A correction regarding my report on books at Rainier Beach: I found my younger son using one of our old homeschool books, PUBLISHED IN 1993, World Cultures: A Global Mosaic, a text where I learned Saddam was still alive and in power and Myanmar was still Burma. This is the history text used in his class and I bet you'd agree with me that this text is way out-of-date. No wonder teachers choose to use their own curricula. Perhaps the money being donated (or if money is available in the budget) should be applied to new history books. (As for my copy, I bought it at a thrift store bag sale, $1.50 a bag.)
Whatever happened to the 2005-2010 five-year plan complete with action steps, due dates, calculated costs and who was accountable? The new plan does not seem to have implementation dates, goals and controls, action steps, whose responsible and estimated cost. Or did I miss something? I usually do.
By the way, Charlie is right, the new math curriculum is ridiculous. We are still using Saxon in addition to the new math program. (Side note, a cost comparison shows that Saxon is cheaper than the books they use now.)
Somewhere, I read that the district was paying for outside tutoring services. I know of two tutoring programs that use students from Seattle University (they need service hours and tutoring fills that requirement). NO MONEY EXCHANGED.
For AP programs available at all schools, I'm not sure if it is possible, but what about partnering with some of the online public high schools? Internet Academy, for instance, offers AP classes as do several others. Think of the cost savings: no need for books because most books are online and no need to hire more teachers because teachers are already in place and offsite.
I'M NOT TRYING TO HIJACK THIS SITE. Maybe a new thread regarding text books being used in Seattle District. Together, I bet we'll see some interesting interpretations.
Why do I keep bringing up this letter? Why should anyone care what the National Research Council (NRC) Board of Testing and Assessment (BOTA) says? BOTA is a premier expert panel, and was formed by the National Academy of Sciences (NAS). Anything published by the NAS or NRC is regarded within the scientific community as high quality and credible. Anything published by NRC is allowed to be cited in manuscripts submitted to peer reviewed journals. The NAS panels are charged with making policy recommendations to the US Govt, based on best-available science. It is because they are asked to write policy recommendations that these panels have many experts, and the experts try their best to reach consensus on their synthesis of the research literature and their policy recommendations.
Do you understand now why a letter from BOTA about RTT and NCLB is so significant?
What BOTA says should be taken very seriously. I searched for a reference to this letter on the ed.gov site today (it is a U.S. Dept. of Education site) with the string "BOTA 2009". Only three records were found, and none of these were the letter to Arne Duncan.
This particular letter was reviewed by an independent panel of experts prior to publication (thus, it was subjected to peer-review).
Not only is the U.S.Dept. of Education ignoring this letter, but so is the press. I don't understand this at all. (Actually, I think I do understand - the press is not so free anymore.)
The writers on this blog (those that write posts, I mean) also are ignoring this letter.
Here are a couple quotes
"The proposed regulations [on the Race to the Top fund of the American Recovery and Reinvestment Act of 2009] rely heavily on the use of testing and assessment both to drive and to measure education reform. BOTA strongly supports the Department's desire to incorporate objective measures of student performance to inform the process of educational reform [and to] help educators and policy makers decide how to target resources."
[Notice that they do NOT say to use such for making important decisions about students and
teachers.]
"...Educators understandably try to align curriculum and instruction [to the test]..., For these reasons, AS RESEARCH HAS CONCLUSIVELY DEMONSTRATED [my caps], gains on high-stakes tests are typically larger than corresponding gains on concurrently administered ‘audit' tests, and sometimes they are much larger. [I can supply an example reported in news media earlier this year.] Improvements on the necessarily limited content of a high-stakes test may be offset by losses [due to curriculum narrowly focused on test content] on other, equally valuable content that happens to be left untested."
How can anyone read this BOTA letter, and, if they understand it, and if they understand the prestige of this panel and the credibility that such panels have within the scientific community, continue to support data-driven decision making and performance management system implementation in SPS? It's beyond my ability to understand.
Read this report folks! It calls in to question, by implication, just about everything MGJ is doing- and just about everything called for in the E4A.
THIS IS NOT BEST PRACTICE!!!! This is really important to understand!!!
The authors of the letter suggest a modification to the proposed regulations, in order to bring it in line with the best available science:
"We encourage the Department to pursue vigorously the use of MULTIPLE INDICATORS [my caps] of what students know and can do. A single test should not be relied on as the sole indicator of program effectiveness. This caveat applies as well to other targets of measurement, such as teacher quality and effectiveness and school progress in closing achievement gaps. Development of an appropriate system of multiple indicators...should be constructed from a careful consideration of the complementary information that is provided by different measures."
This is why I have said in other postings that I have no problem with MGJ failing to meet her deadlines for implementation of E4A. The further behind the better. The less damage she will do while she is here. The sooner we get rid of her the better. The sooner the Board shapes up and starts doing their job to serve the best interests of students and teachers (rather than serving best interests of business people looking to make some bucks), the better.
We just have to be very careful that we don't get another Superintendent who will continue the job MGJ started.
We also have to do our best to prevent Mike McGinn from winning the power to appoint directors. If he gets that power, then we will eventually be in worse shape than we would have had he not gotten this power. He'll pick Board members who will practically give away valuable publically-owned property to friends and acquaintances who want to start their own schools. (I wouldn't be surprised Nick Hanauer will be one of those who benefits.)
We need to get campaign contribution limits for the school Board directors. There is currently NO LIMIT! Did you know that just 18 very wealthy people donated nearly a quarter of a million dollars to school board campaigns in 2007? DO the math: that's an average contribution of about $14,000 each. (Nick Hanauer, venture capitalist and founder of League of Education Voters, gave nearly twice that much.) The money went to four candidates: Carr, Maier, Sundquist, and Martin Morris (M-M got the least of these four, but still, he got about about $65K to Sally Soriano’s $13K).
We have a better chance of getting true advocates for students, families, and teachers on the Board (e.g., Sally Soriano, Mary Bass, and others) if BIGGGG money wasn't influencing the outcome of the school board elections. The Mayoral campaign limit is $700. The School Board limit should be lower than this - say $500.
[I don't agree with Charlie that we would be likely to have better results with an elected superintendetn accountable to a Board appointed by the Mayor.]
To summarize: To those of you who are upset that MJG isn’t being punished for falling behind on the schedule for implementing E4A, I say,
“PULL YOU HEADS OUT OF THE SAND!”
which is to say, "Read the BOTA letter (at least the first several pages, anyway)!" And then give copies of the letter to School Board Directors every time you see them. Don't let them ignore this letter.
1. I don't exactly get your point and its relevance; please elaborate.
2. Please read the first several pages of the BOTA letter, and tell me if you agree or disagree with my attaching so much significance to it.
As for campaign contribution limits, I don't see it happening anytime in the near future. I have to fight the battles now that I know I can win (or at least find a way around so they don't apply to my child). Have your children assessed outside the system, but make sure to maintain a portfolio of their work. Grades, however, do matter.
My point about the rise in remedial colleges at community colleges demonstrates much better than any standard district tests that our schools have failed.
My two children (one in 1st grade; one in 3rd grade in Lowell APP) are high achievers, and I don't anticpate them having any test anxiety. My concern is not about my own children. I have an egalitarian, humanist, democratic, anti-racist orientation. I most certainly want my children to be successful, but I put a high value on trying to help children less fortunate than my own be more successful in their lives.
Standardized testing and high stakes testing won't, in practice, won't harm MY high achieving children. Nevertheless, I still am opposed, in principal, to high stakes testing. The BOTA letter shows that high stakes testing is scientifically unsupportable, and should be avoided.
DATA DRIVEN DECISION MAKING is an approach to public education that includes high stakes testing and performance management (wherein decisions about teacher pay, tenure, and promotion is based on students' test scores) as its central methods. For this reason, we must reject DDDM for the same reason that we must reject high stakes testing.
DDDM, high stakes testing, and performance management are not good for our students, our teachers, our schools, our community.
What is truly remarkable about advocates for DDDM is that their desire to expand its application reamins unabated, in spite of overwhelming evidence that DDDM has had little or no effect on the achievement gap narrowly and unsupportably defined by high stakes test scores. Furthermore, DDDM has not been successful in improving on-time high school graduation rates, post-graduation job-readiness, and post-graduation college-readiness.
DDDM is NOT about best practices in education, despite what the advocates say.