Thursday, August 06, 2015

Washington State Board of Ed Struggles with SBAC Cut Scores

Update: Here's what the BOE had to say about yesterday's meeting and its outcomes. I think BOE head, Ben Rarick, has done a very good job in explaining the Board's thinking.

The Board followed through on its equal impact philosophy, adopting a mid-Level 2 score requirement for the English Language Arts (ELA) portion of the Smarter Balanced (SBAC) assessment (scale score: 2548), and followed the same philosophy for the Math End-of-Course exams. The SBAC math score (2595) was set to be commensurate with the ELA requirement. These minimum scores are just a little more than half way up the Level 2 scale; about 60% of the way between Levels 2 and 3.
The Board wanted me to help explain their decision to you all, and emphasize a few points we can all work on together for the betterment of students.

First, the Board wants to emphasize that Level 3 remains the goal for all students on the new (SBAC) assessments. A Level 3 score represents a career and college-ready score for our students. The Board wishes – indeed expects – all students to eventually be able to achieve this level of proficiency. Although the board has set a transition standard at a rate below Level 3, this was done to ease the transition for our system and demonstrate fairness to students. It was not done to compromise or confuse our ultimate goal.

Second, as exciting as the 10th grade results were, the results from juniors on the SBAC were perplexing. Fewer than half of juniors took the assessment, and those who did were greatly surpassed in achievement by their sophomore counterparts. The sophomores outperformed the juniors to such an extent that it is obvious that something is wrong. As a result, the Board was limited in its ability to use this data to set scores for the math SBAC.

Finally, and perhaps most importantly, while these assessments are important to our goals, but they are not the goals themselves. There is a difference between taking these assessments seriously, and letting a test define a student. No test defines a student.

Let’s acknowledge the important role that SBAC assessments play in career and college-readiness, without letting them become the definition of career and college-readiness. Kids are so much more.

end of update 

I was following the tweets of ace ed reporter, Kyle Stokes of KPLU, yesterday as he sat thru the slog that was the State Board of Education meeting over figuring out cut scores for the high school SBAC test.

The struggle?  So many 11th graders didn't take it that they could not figure out where to cut.  Ann Dornfeld of KUOW reported this morning that the BOE even thinks that of the 11th graders who did take the test, many of them did not seem to be trying all that hard.

So the BOE went with a lower score with an eye to try to raise that score when they get more data from the next couple of years.

Except - what if 11th graders continue to opt out in large numbers?  Quite the dilemma.  And will it be the carrot or the stick to try to get students (and their parents) to buy into the value of SBAC?

What's that old saying? Power to the people, right on.

Links to come as soon as the stories from Stokes/Dornfeld come on-line. 

17 comments:

Anonymous said...

"The sophomores outperformed the juniors to such an extent that it is obvious that something is wrong. As a result, the Board was limited in its ability to use this data to set scores for the math SBAC."

Could that "something" be that Sophomores didn't take the SBAC Math test?


10th grade parent

Maureen said...

Fewer than half of juniors took the assessment, and those who did were greatly surpassed in achievement by their sophomore counterparts. The sophomores outperformed the juniors to such an extent that it is obvious that something is wrong. As a result, the Board was limited in its ability to use this data to set scores for the math SBAC.

Why on earth are the low 11th grade scores surprising? Clearly, (1)the kids who would have done better were more likely to opt out and (2) the kids who did take it knew it served no purpose and generally didn't try very hard. Are the people in charge being purposely disingenuous, or do they really not understand this?

Carolyn Leith said...

It's interesting Ben Rarick wants to walk back the claim that the SBAC assessments are the definition of career and college-readiness.

"A well-rounded student who is truly “career and college-ready” is more than simply proficient in Math and Language Arts. Let’s acknowledge the important role that SBAC assessments play in career and college-readiness, without letting them become the definition of career and college-readiness. Kids are so much more."

According to the website for the Common Core State Standards Initiative, the standards, which the SBAC evaluates, ARE the definition of what is college and career ready.

"The standards were created to ensure that all students graduate from high school with the skills and knowledge necessary to succeed in college, career, and life, regardless of where they live."

You can read more about the early efforts to market the common core standards as the tool to achieve college and career readiness on Mercedes Schneider's blog.

Sorry, Ben, you don't get to revise history now that the public is starting to pay attention.

Anonymous said...
This comment has been removed by a blog administrator.
Anonymous said...

11th graders will be motivated to take the ELA SBA since they need it for graduation, but you're right that 11th graders might continue to opt out of the math portion for the next 2 years (since this year's 9th graders are the first for whom it's a requirement). If you're already taking the ELA portion of the SBA, that might have a spillover effect and reduce opt-outs on the math portion somewhat as well. But I suspect many of those kids will be savvy and just take the ELA piece, even if schools make it hard to pick and chose like that.

As a result, the SBE will likely get more data, but they will be more skewed. While none of last year's 11th graders had any incentive to take the SBA exams, for the next two years it seems likely that 11th graders who do opt to take the math portion will be a very unrepresentative group. Students who have already passed a math EOC won't need the SBA, but 11th graders who don't already have a passing EOC math score may want to take it to maximize their chances of meeting a requirement. This means 11th graders who do take the math SBA are likely to be those who are weaker in math--which would likely make it trickier for the Board of Ed to use those data in adjusting cut scores. I suppose they could figure something out (e.g., if the 11th graders who take and pass the EOC that year are not also passing the math SBA, maybe they've set the cut point too high), but if the picture isn't clear they'll likely need to wait until they have a full year's worth of "good" data. If today's 9th grade cohort does so poorly on it in 11th grade, maybe they could lower the cut score at that point, retroactively. They won't want a much larger percentage of students failing to graduate just because they messed up on their score setting, right?

HF

Lynn said...

A high percentage of 10th graders took the ELA exam this year. Those whose scores met the level required for graduation will have no reason to take it again in their junior year. Those who didn't score high enough now qualify to use one of the other standardized test scores for the graduation requirement. I hope opt out rates increase next year. Let's not provide any of the data they want so badly on this experiment.

Lynn said...

A high percentage of 10th graders took the ELA exam this year. If their scores met the graduation level they have no reason to take the exam again. If they didn't score high enough, they now have the option to use other standardized test scores to meet the graduation requirement. (AP/IB/ACT/SAT) I hope opt out rates are higher next year. Let's starve them of the data they need for this experiment.

Anonymous said...

Huhh??? How bizarre.

This data fantastic trip generates "cut scores" not based on any desired demonstrated competency of content but rather what the group did.

"A Level 3 score represents a career and college-ready score for our students. The Board wishes – indeed expects – all students to eventually be able to achieve this level of proficiency.

If so the Board should be dismissed, as only a person completely ignorant in regard to data and human variability would have such a "one-size fits-all level 3 expectation".

The "Career and College Ready" graduation requirement will be the new "school to unemployment pipeline".

To improve a system requires the intelligent application of relevant data.
This Board expectation is an epic failure. Common Core + SBAC + Board = idiocy.

-- Dan Dempsey

Anonymous said...

@ Dan, I doubt that's a real expectation on the part of the SBE. It's a wish, clearly. And "eventually" achieving that level may mean in some hypothetical future, where we've solved problems of poverty and racism, where we have awesome special ed services, etc. But if they aren't seeing higher student scores over time, I doubt they're going to keep raising the cut scores so that more and more students fail to meet them. They're anticipating that scores will increase and that cut scores will be adjusted in accordance with that. No need to try to panic people. (And actually, isn't the old system already based on what you call a "one-size-fits-all level 3 expectation"? Doesn't passing the HSPE mean scoring at least level 3, proficient?)

@ Lynn, I get that you don't like the SBAC, but I have a hard time seeing people advocate that kids who have not yet met graduation requirements actively take steps to minimize their options to do so. Yes, if they failed the ELA SBA this year they have other options. But don't assume that it'll be easy for those kids to get a high enough AP or SAT score, or that they're prepared to go through the hassle of collection of evidence or whatever. Skipping the test just to make a point could really backfire on some kids.

HF

Anonymous said...

HF-

Charlie once told me about aspirational goals... (not real goals but ones we will pretend are real goals for rhetorical purposes)

Check this out from the SBE contact us page:

The Washington State Board of Education provides leadership in the creation of an educational system that personalizes education for each student and respects diverse cultures, abilities, and learning styles. The Board does this through advocacy and strategic oversight of public education, implementing a standards-based accountability system to improve student academic achievement, and promoting achievement of the Basic Education Act goals of RCW 28A.150.210.

I am definitely missing the SBE respecting diverse abilities.

One size fits all graduation diplomas produce a fraudulent system.. notice the fake goals expected for all.

-- Dan Dempsey

dan dempsey said...

HF wrote:

that they're prepared to go through the hassle of collection of evidence or whatever

I believe the legislature might have ended the collection of evidence in the most recent session.

Did HB2214 pass into law?

Anonymous said...

@ Dan, you may be missing the "respecting diverse abilities" part, but I'm also missing the personalized education and diverse learning styles parts. So maybe when they say they are providing leadership in the creation of such a system, they are actually acknowledging that it doesn't yet exist! :)

I recall you've advocated for a range of diploma types, indicating different levels of achievement. I'd be fine with that, but don't really see the need. Maybe a certificate of "completion" or something for kids who attend the required classes but don't get the scores they need would be nice, although I'm not sure how that would really benefit them in the long run. Is an employer more likely to hire you if you finished high school but didn't graduate than if you didn't finish? Hard to say--it could just as easily backfire. As for the levels of achievement for kids who meet or exceed the standards, why does a kid who does better need a higher level diploma than someone else? Whether they go off to college or get a job, admissions offices and employers can see their grades, classes, test scores, etc. and treat them accordingly. The current requirements are a floor, not a ceiling--the diploma may be one-size-fits-all, but the classes taken en route to it are not. While many are concerned with increasing the rigor, you seem to want to lower the floor? I'm obviously not understanding your thinking on this.

If I recall correctly, you also dislike the math requirements and think it's ridiculous to expect that everyone can do algebra. You cite rates of algebra EOC exam success and such as evidence that it's just too hard. I have to disagree. Just because we are not currently doing a good job at educating kids in math does not mean it's impossible. Take a good math teacher and give them sufficient time with most kids and I'm confident things would finally click. The levels of math that we're talking about really are not all that hard, if you get a strong foundation and strong teaching throughout. Can we get there anytime soon? Probably not. But should we just throw our hands up in the air?

HF

Anonymous said...

Is an employer more likely to hire you if you finished high school but didn't graduate than if you didn't finish?
Yes! Finishing something, with or without passing a certain test is important! Finishing something, perhaps without absolute mastery is also something important. This is not hard to say at all. Yes. We already do have differing certificates backing dimplomas. CAA, CIA, CA. Isn't that enough? This sort of detail really isn't widely known, and is fairly irrelevant. Standardized rigor, by itself, is not all that interesting. It's much more important that students have the option to develop talents that differentiate you from the pack, and that provide fuel for further personal development. And right. A high school diploma isn't the end of the road. It's the starting line. It doesn't and shouldn't mean too much. For high achievers, it will mean very little, as it always has. For those who struggle, it should be be an attainable hurdle. There are an increasing number of options for low achievers, and the high school process should not be designed to impede that.

Sped Reader

Anonymous said...

So, what is the "cut score" for students with disabilities passing with MO(dified) IEP accommodation? Normally, the MO accommodation allows students with disabilities to pass with anything in the 2 range. If the "cut score" for regular achievement is 2.5, the "cut score" for students with disabilities taking the MO option should also be commensurately lower, like a 1.5. Now we have the situation where a regular pass is essentially a "MO", but students with disabilities need more support and more accommodations than others. What then is the MO-MO (since the regular test is already MO)? Predictably, the SBE didn't appear to address this discrepancy - at all. Once again - students with disabilities are not considered - at all, in the common core debacle. Sure - students with disabilities can also take the SBAC at an "off-grade" level, but what is the "pass score" for those taking the test off-grade level? If those taking the regular test and receiving a "cut score" of 2.5 - do students with disabilities taking an offgrade level test also get the same reduction in "cut score"? Eg. Could a high school student taking the 8th grade ELA SBAC get the same "cut-score" of 2.5? If not, this is actually unfair for students with disabilities, because the bar has been raised for them alone. Eg. They have to still meet the additional burdens of additional standards, and meeting standards that were never taught in their special education classes - but get no breaks from the "equal impact" efforts like others. Students with disabilities typically take longer to show mastery, and are more in need of "equal impact" than others, not less in need of it. And finally, this likely means that students with disabilities will have to suffer more and more testing - as they fail various levels. One approach for students with disabilities - take the lowest possible grade level offered, 3rd grade. That will offer the biggest chance of success. It will also be the most meaningless.

Sped Reader

Anonymous said...

HF-

Here is my thinking.
#0.. No certificates of attendance.

#1.. Many students like to aim at a goal. The Advanced Academic Diploma was something that influenced at least one of my students in NV to take harder courses. She wanted that diploma.

#2.. For the General Diploma, I would still require a passing score on the ALGEBRA EoC. The chance of a student, aiming for a general diploma attaining it, would be enhanced because classes that were focused on helping that child overcome low skills and inadequate k-8 math preparation could be provided prior to the Algebra course. Note: the current Algebra EoC requires no competence in factoring or facility with quadratic equations, which may lead to neglecting those "real" algebra topics in Algebra class.

#3.. A Geometry class with greater rigor could be offered than the current offerings many of which are aimed at getting everyone graduated.

#4.. A lot of jobs require a high school diploma. Truck driving at the Robinson Copper Mine requires a high school diploma. Lets stop manufacturing fake achievement because of one size fits all graduation requirements. Fewer "Dumb-down" Geometry classes etc. please. ... The legislature's ending the Collection of Evidence was a good step in the right direction.

#5.. ACT finds 5% of Black students College Ready. NY Judge finds tests not discriminatory. An analysis last year found that 46 percent of Hispanic candidates and 41 percent of black candidates passed the test for a teaching license in NY on the first try, while 64 percent of white candidates did so.

It is long past time to stop the faking and begin the fixing of "faking accomplishment". An excellent next step would be requiring WA HS Diplomas to mean what they say.

-- Dan Dempsey

Anonymous said...

NAEP reaching proficient

Look at the above chart. Clearly requiring proficiency in grade 12 as a graduation requirement for all is never going to happen. Yet the current system masquerades as if a "College and Career Ready" expectation for all is reasonable.

This is such bunk.

grade 12 students rated as "Below Basic" on NAEP Math (2013)

25% - white

62% - black

50% - Hispanic

46% - American Indian/ Alaska Native

19% - Asian/Pacific Islander

==============
Maybe requiring reading skills to advance to grade 4 could be the focus ... rather than the current nonsense college and career ready diploma.
==============

-- Dan Dempsey

Anonymous said...

There really is no need for that complexity in the diplomas. And, it won't happen. There already are different types of certificates backing diplomas for students with disabilities. For college-bound students, college like the old days, they are already getting a college degree and the high school diploma will quickly fade as something irrelevant. If you're looking for more complex geometry class, or more advanced offerings generally, there already are honors sections of nearly every class. So, that wouldn't be the impetus for this multi-tiered diploma. And finally, there's always the transcript. For people really hung up on rigor and "meaningfulness" of the high school experience, the transcript can always prove exactly what you did. There are a million new colleges, and college types - popping up everyday. Viva la difference. Washington State diplomas do mean something. College and Career ready - just don't happen to be what you think they should be.

Reader