Tuesday, November 10, 2009

PSAT Results

It never hurts to ask.

So I had sent a Public Disclosure request for the PSAT results by grade at each school, school, grade level across the district and district overall. I had spoken to the head of Advanced Learning, Bob Vaughn, briefly about this and he said they had not had time to break out any results.

Then I called Boeing. (They paid for it.) Turns out they don't have those results either but are due a report by the end of the year from the district. The guy I spoke to said the results are public data.

The response I received from the district's Public Records was this:

"I am writing in response to your email below requesting PSAT test results. In doing so, I learned that the test results that we receive are in a format that cannot be easily incorporated into our information, which would allow us to release statistical information without violating individual student confidentiality. I am looking into whether it would be possible to redact or remove student identification from the results we get from the College Boards and/or extract statistical totals.

I anticipate being able to give you another response or update on or before December 4, 2009. Please let me know if you have any questions."

I really respect Joy Stevens (who the the district's senior legal assistant and handles public records). I'm sure she get plenty of requests and has her work cut out for her. But I find it hard to believe that they would have formatted them in such a tight fashion that you could not find results in different forms (grade, school, district) without revealing students' names. What use would they really be to the district except as an exercise in getting a large number of students to try this test? Hmmm.

24 comments:

hschinske said...

I would call the College Board and ask if it's true about the format.

Here is an example of a report on PSAT data by schools: http://www.houstonisd.org/ResearchAccountability/Home/Perform_Acount/StudPerf/PSAT%20Reports/Reports/PSAT_2009.pdf

If they can do it, so can we.

Helen Schinske

dan dempsey said...

I agree with Helen:"If they can do it, so can we."

Right ... does anyone believe the SPS on this one???

Test was given in October 2008. Results were reported to high schools in Jan 2009.

WOW!!!! what kind of spreadsheet work does it take to delete the student name column and sort the rest? Does it really take more than 10 months?

Boeing might be moving everything to South Carolina if the SPS is really this incompetent.

Patrick said...

This sounds like an episode of Yes, Minister.

dan dempsey said...

This takes us to the larger question of outright deception by the administration.

When teachers were RIFed was the board aware that close to $11 million was going to be used for 111.5 coaches?
That is why more teachers cannot be afforded.

The SPS adopts math programs that are so user unfriendly that expensive professional development is needed and still the results are awful.

At the time of the "Discovering" High School adoption was the board aware that a good portion of those 111.5 coaches would be needed for math professional development ... say maybe 4 to 5 million dollars worth annually?

DeBell got it right on Thursday 11-05-2009 with:
"Is there any evidence this coaching model works?"

Where is the data???

To improve a system requires the intelligent application of relevant data.

Seems pretty apparent that TEAM MGJ hides lots of data ... both academic and financial. But does the board even care? Accountability??

SPSMom said...

I have always wondered if the board had known that they were going to have to approve $500K to purchase more EDM consumables BEFORE the Discovery Math vote if the outcome would have been different. Maybe one director would have said, "wait a sec, we need to look at the big picture here and that picture is expensive." The board is JUST now learning that millions of dollars are spent on coaches? Really hard to believe, but appears to be true and this only came to light as a result of Meg's report. (and now it appears another parent has enrollment numbers for Laurelhurst that basically says that Sandpoint may not need to be opened or if opened really needs to be an option school.)

Point is, the board is only given information needed to get the response the district is looking for, the board never has the complete picture when making these huge decisions that will impact thousand of students for years to come.

You have give MGJ credit, she really does know how to "manage" her bosses.

And to all you parent data wonks THANK YOU!

ArchStanton said...

WOW!!!! what kind of spreadsheet work does it take to delete the student name column and sort the rest? Does it really take more than 10 months?

Maybe it's on the VAX?

dan dempsey said...

"You have give MGJ credit, she really does know how to "manage" her bosses."

Seems like some of her bosses enjoy being managed rather than thinking.

The easy way is just to rubber-stamp everything.

Often it seems as if some directors statements are written by Central Administration.

Dorothy said...

I'm confused. Does this mean they give Boeing the information with names? I didn't agree to that. Why can't we get the same report Boeing does?

What I care about more than the actual numbers for each school is data on consequences. Outcomes. How many students that scored on the low end got extra intervention and how many students who scored on the high end got noticed and encouraged to take more challenging classes and/or were informed about outside opportunities (summer programs, scholarships...) And did any student take advantage of that information that would not otherwise. What was the goal of the grant and what was the effect?

dan dempsey said...

A little history on this:

Date: Monday, February 2, 2009, 1:09 PM

Hi Dan,

PSAT was administered through Advanced Learning, so I’m forwarding your question to Bob Vaughan, who heads that department.

Anna-Maria
--------------

Anna-Maria also cc: Carr and Martin-Morris.
--------------
Months later I let Sherry know that I never received the PSAT results.

================
Needless to say the job of school director is darn difficult given the level of deception practiced by Team MGJ.
================
Thanks to Meg.
111.5 coaches for 10 million+...
at last I know where the I-728 money went ...well maybe....

for MGJ said all the coaches were paid for with grant money at the time of the RIFs

Who do you trust?

Me....I trust Don Kennedy to spew the Central Admin line.

Keep asking questions Michael DeBell and looking for answers.

Like .. why you were not informed of all this when you were trying to reduce the number of RIFed employees on June 3, 2009 .. Mr. Kennedy's report at that board meeting now looks woefully inadequate.

dan dempsey said...

Dorothy,

Boeing has given $$$ to several local school districts with few strings attached ... trusting the school districts to do what is right.

I agree how are the outcomes of this testing used to help kids and improve program?

Given that the district has completely failed to follow D44 & D45 in providing effective interventions for struggling students k-8, I would not be putting much hope in them doing so with the PSAT information.

Hey but maybe they did. Perhaps a Seattle High School teacher or administrator could enlighten us.

seattle citizen said...

OT, but I just saw that right now (7:00-8:30 pm, Tuesday)
on Channel 21 SEA (comcast) is a panel discussion on youth violence in Seattle. Police, officials, gang members (ex?) teens....

Melissa Westbrook said...

I think we will be able to get the report that Boeing gets but I would assume Boeing gets data without names. It's a good thing to ask.

Josh Hayes said...

I tell you what, I have experience writing SQL queries on a variety of platforms -- including VAX/VMS, God help me! -- and I'd be happy to volunteer, free for nothing, my time to generate the data Melissa's asking for.

Assuming the database is somewhere on District servers, it should take about an hour -- 55 minutes to slog through the tabs and categories, and five minutes to write the query.

If they can't do this with current in-house staff, they need to fire current in-house staff and hire some people with database proficiency. I'm a biologist, for god's sake, and _I_ can do this. Harumph, I say! Harumph!

dan dempsey said...

Let me guess if and when this is ever done the PSAT median Math scores for 9th grade and for the 10th grade will not be very good.{and likely 9th scores and 10th scores will indicate not much growth in one more year of SPS math education} If those median scores were great, it would have been in the media very quickly. {Just guessing from past practice}

This should be a good baseline for the SE Initiative. Is the Central Admin ever planning to measure anything and correlate it to a national scale (and publicly disclose the results)?

Cheryl Chow's emphasis on "Public Relations" is in full swing no doubt. Really hard to improve when you have no idea where you are.

Melissa Westbrook said...

And that's the thing; this is the first time the district has ever given the PSAT, in masse, across 3 grade levels. I had no expectation that we would see great or even good results. I just thought it would be a baseline of where we are and what we need to do to get better.

Maybe we are overthinking this but it just seems odd to not just take the data and lay it out for all to see.

taylor said...

We never got the data for my child's test in 10th grade...and after waiting for months, my child took another college entrance test, the ACT, and have used this to get into college two years early.

It was a far, far better thing to drop-out of Seattle Schools at the end of 10th grade, 2 years early, and enter college than to wait for the district to catch up to my child or stay waiting for the child to underachieve By the way, the scores were never handed out to each child BUT are in each child's guidance file with high school guidance counselors.

I hear across the board that these PSATs were so very stinky that guidance counselors did not want to hand out the scores for fear of fielding tons of calls ...and more importantly because the district would have to put additional $$$'s behind helping students to make future progress on future SATs.

The deal is if you identify a problem... then you are held responsible for fixing it. Seattle does not have the real political will or financial backing to fix this systemic problem that will keep our children from making it into college... Don't wait until 11th grade to figure this out for it will be too late!

dan dempsey said...

Taylor said:
"Don't wait until 11th grade to figure this out"

When the district has a "de facto" decade plus long policy of social promotion in violation of their own published policies... they do not want anyone to figure it out....

For there is no accountability.

Charlie Mas said...

My daughter was one of the ninth graders who took the test last year. We received her scores from the College Board (or whoever it is that publishes this test) in a timely manner.

My understanding was that the reason for having every student take the test was a sort of talent search, to find students who weren't thinking about taking AP classes or thinking about college but perhaps should consider them.

I didn't think that the test was supposed to be an assessment for schools or the district. I didn't think that there was going to be any sort of roll up group reporting and I wasn't looking for it.

What I WOULD like to see, however, is a report of how many students were encouraged to take more challenging courses as a consequence of their test results. I would like to see some evidence that the test served its purpose.

dan dempsey said...

Charlie,

When the district spends this much cash or Boeing cash, it seems nonsense not to use the results as Melissa suggests as a baseline for comparison in the future.

Also when the district looks to see if k-8 math is working would not grade 9 PSAT Math results give some indication?

That is one reason I do not think the Central Admin wanted these results available to the board before the High School math adoption.

Do you have any idea how the individual results were used thus far?

Sounds like if a student did poorly they may have not even received results ... so your thoughts about identifying the talented may be right on. Was it: Identify the talented and ignore the rest?

hschinske said...

taylor, given that your child did well enough on the ACT (and presumably in many other respects) that s/he got into college two years early, it's entirely possible that the PSAT scores could have been in the National Merit range. (I know more than one student who scored that high on the 9th-grade exam.) I would contact the College Board directly.

It is possible, if you jump through the right hoops ahead of time, to have 9th- or 10th-grade scores counted for National Merit purposes if the student enters college early. You won't be able to do it now, after the fact, but you'd have ammo to complain to the district that your child would have been a National Merit semifinalist or commended student, and they threw away that chance.

Helen Schinske

seattle citizen said...

OT again...
I watched the special on Seattle youth violence last night. What the youth said, loud and cleas: They need similarly-aged people, who can speak and listen in their language, who are motivated and moving into the adult world, to mentor ans support them, be there, take them to lunch...They need adults who care, who participate in the community, who don't put up with some of the crap out there.
My apologies to any of the particpants if my paraphrase is off; this is what I felt. Good job to all, some very powerful speakers, particularly the younger ones who are out there every day.
Here's some interesting articles in today's NY Times:
A Parent’s Unemployment Stress Trickles Down to the Children
"Several academic studies have linked parental job loss — especially that of fathers — to adverse impacts on everything from school performance to self-esteem."
http://www.nytimes.com/2009/11/12/us/12families.html?_r=1&ref=us

States Compete for Federal School Dollars
http://www.nytimes.com/2009/11/11/education/11educ.html?ref=us
"Wisconsin is also competing. Gov. James E. Doyle has backed several education bills recommended by the Obama administration, including one that would remove a prohibition on using student data to evaluate teachers, which the administration has made an eligibility requirement of the grant competition. President Obama traveled to Madison, Wis., last Wednesday and urged lawmakers there to revoke the prohibition, praising California and Indiana for recently taking the same step. By week’s end, Wisconsin lawmakers had done so, too."

25 Chicago Students Arrested for a Middle-School Food Fight
http://www.nytimes.com/2009/11/11/us/11foodfight.html?ref=us
" 'They were handcuffed, slammed in a wagon, had their mug shots taken and treated like real criminals.'
'They’re all scared,' Ms. Russell said of the two dozen arrested students. “You never know how children will be impacted by that. I was all for some other kind of punishment, but not jail. Who hasn’t had a food fight?'
The students were released into the custody of their parents on Thursday night, the police said. They were also suspended for two days by the school, the Calumet middle-school campus of Perspectives Charter Schools"

Melissa Westbrook said...

I've never had a food fight.

seattle citizen said...

I haven't either, but there were food "shenanigans" in our cafeteria, middle AND high...

Now I just "fight" over what's for dinner! (and because of expense, we hardly ever eat at WV's recommended "canelese")

seattle citizen said...

Back to the PSATs...I agree with Melissa and others that these could be benchmarks, particularly since the 10sa taking it this year took it last year...what progress? And if it were continued for 9s (it's not, alas, no more grant and it IS $13 a pop) it could be used to benchmark rising or falling scores at 9 over time...
And it's good prep for SAT/ACT. I wonder if those who can afford it will start paying for test-prep classes for the PSAT, as some do for the SAT? I took the GRE twice, because I was interested in the social justice aspect of the different scores between those who couldn't afford prep and those who could...I was interested in how much a difference it made (I was on an assessment study group at college for a year when I did this)
The $400 course improved my score about 25% over the test I took after just cursory review of a test prep book.