Assessment Task Force Update
I mentioned previously that I had attended the Assessment Taskforce meeting on Feb. 21st.
I wanted to make note of several things I noticed:
- Organized - very much appreciated
- Good facilitator who kept things on track
- The group is going to be able to look at the MAP test. I think this is great because as members of this Taskforce, they need what they are talking about when it comes to this discussion.
- Two principals - from Mercer and Denny - came in to talk about what was happening at their schools. A little bit of cheerleading there for MAP but I think the Taskforce took it with a grain of salt. The only odd thing was the Denny principal saying they had a data wall with kids' scores on it. I hope not.
- As in any group, there are a few people who speak often and the rest listen. I hope that the listeners don't allow the discussion to always follow what the speakers want it to be. That's a facilitator's job.
- The head of Curriculum and Instruction, Shauna Heath, says there is nothing out there like MAP for "validity and reliability." Hmm.
- Kris McBride from Garfield was a calm and firm voice about how MAP does not work for their students. She did not question its use for other grade levels but said it was not helping for the high school level.
The good, best thing at that meeting?
They had Linda Darling-Hammond of Stanford University, a very noted and well-respected education expert, call in. Dr. Darling-Hammond (unbelievably) listened to the entire meeting before her own section started. I was surprised she had the time but I think she wanted to know what was being discussed.
She brought forth the following from the paper she co-wrote with Frank Adamson, Beyond Basic Skills: The Role of Performance Assessment in Achieving 21st Century Standards of Learning.
I urge you to read this paper. I found it incredibly useful. I recommend pages 17-22 where she talks about what other states are doing. From the preface:
Whether the context is the changing nature of work, international competitiveness, or, most recently, calls for common standards, the premium today is not merely on students’ acquiring information, but on recognizing what kind of information matters,why it matters, and how to combine it with other information.
What she had to say at the meeting:
- MAP has some utilitarian uses but its structure is around what is currently taught and not what is coming i.e. Common Core
- She worked on Obama's transition team and one thing he emphasized was that testing needed to move beyond multiple choice questions.
- She said the US is pretty out of step with what other countries are doing BUT is getting there. Her words at the meeting and her paper both reflect that high-performing countries are using open-ended questions, oral questions and essay/project-based assessments. (I could see the senior project become a sophomore or junior project that has more assessment weight.) For example, in Victoria, Australia, on-demand tests are supplemented with classroom-based task, given throughout the school year, that compromise at least 50% of the examination score.
- She said it was important that assessments represent higher learning and asked two questions.
Do the questions demonstrate how students will succeed in tasks in college and career?
Does it give information to teachers on how they learn and not just how they score?
She said to think about countries where the depth of the questions is more important than a "quick and easy" scoring ability.
- She was asked about MAP and a Rand study of 17 states including Washington State. There was a ranking of the questions 1-4 (with 4 being the best) and found that 0% of the multiple choice questions reached a 3 or 4.
- She mentioned Conn., NY, Vermont as states with districts that have developed portfolio strategies that can be scored in reliable ways, across grade levels.
I hope to be able to attend tomorrow's meeting but I was very impressed with the discussion that went on at this meeting.
I wanted to make note of several things I noticed:
- Organized - very much appreciated
- Good facilitator who kept things on track
- The group is going to be able to look at the MAP test. I think this is great because as members of this Taskforce, they need what they are talking about when it comes to this discussion.
- Two principals - from Mercer and Denny - came in to talk about what was happening at their schools. A little bit of cheerleading there for MAP but I think the Taskforce took it with a grain of salt. The only odd thing was the Denny principal saying they had a data wall with kids' scores on it. I hope not.
- As in any group, there are a few people who speak often and the rest listen. I hope that the listeners don't allow the discussion to always follow what the speakers want it to be. That's a facilitator's job.
- The head of Curriculum and Instruction, Shauna Heath, says there is nothing out there like MAP for "validity and reliability." Hmm.
- Kris McBride from Garfield was a calm and firm voice about how MAP does not work for their students. She did not question its use for other grade levels but said it was not helping for the high school level.
The good, best thing at that meeting?
They had Linda Darling-Hammond of Stanford University, a very noted and well-respected education expert, call in. Dr. Darling-Hammond (unbelievably) listened to the entire meeting before her own section started. I was surprised she had the time but I think she wanted to know what was being discussed.
She brought forth the following from the paper she co-wrote with Frank Adamson, Beyond Basic Skills: The Role of Performance Assessment in Achieving 21st Century Standards of Learning.
I urge you to read this paper. I found it incredibly useful. I recommend pages 17-22 where she talks about what other states are doing. From the preface:
Whether the context is the changing nature of work, international competitiveness, or, most recently, calls for common standards, the premium today is not merely on students’ acquiring information, but on recognizing what kind of information matters,why it matters, and how to combine it with other information.
Remembering pieces of knowledge is no longer the highest priority for learning; what students can
do with knowledge is what counts.
What she had to say at the meeting:
- MAP has some utilitarian uses but its structure is around what is currently taught and not what is coming i.e. Common Core
- She worked on Obama's transition team and one thing he emphasized was that testing needed to move beyond multiple choice questions.
- She said the US is pretty out of step with what other countries are doing BUT is getting there. Her words at the meeting and her paper both reflect that high-performing countries are using open-ended questions, oral questions and essay/project-based assessments. (I could see the senior project become a sophomore or junior project that has more assessment weight.) For example, in Victoria, Australia, on-demand tests are supplemented with classroom-based task, given throughout the school year, that compromise at least 50% of the examination score.
- She said it was important that assessments represent higher learning and asked two questions.
Do the questions demonstrate how students will succeed in tasks in college and career?
Does it give information to teachers on how they learn and not just how they score?
She said to think about countries where the depth of the questions is more important than a "quick and easy" scoring ability.
- She was asked about MAP and a Rand study of 17 states including Washington State. There was a ranking of the questions 1-4 (with 4 being the best) and found that 0% of the multiple choice questions reached a 3 or 4.
- She mentioned Conn., NY, Vermont as states with districts that have developed portfolio strategies that can be scored in reliable ways, across grade levels.
I hope to be able to attend tomorrow's meeting but I was very impressed with the discussion that went on at this meeting.
Comments
Maybe the difference is assessment for basic skills vs. skills we think are important to see in a HS graduate. If we think knowing the basic R's is the minimum expectation of a HS diploma (and leave out the part about pulling it all together and using that basic knowledge to develope higher knowledge), then the present system of low quality perfomance testing is probably good enough and more affordable. It certainly would be cheaper for testing companies to use scanners and computers than to hire people to read essays and short answers.
off track
In my child's middle school class, the teacher lifts some assignments from random websites [out and out plagiarism]. The students are kept busy with time consuming projects that require little in the way of deep thinking. I'd be happy if there was even one essay a month.
It's the curricula and the teaching. Without strong curricula, the weaker teachers seem to flounder.
ranter
Hear, hear. I've been trying to make this point on the Strategic Plan Taskforce as well.
But I get the impression that some members of the district staff (and the facilitators themselves) believe that teachers alone determine the quality of the classroom experience.
They are definitely a big part of the equation, but I think that's too narrow a view, and puts all responsibility for outcomes solely on the shoulders of teachers.
If anyone on this blog or beyond feels that SPS should address improving curricula as well as teaching as we move forward, I urge you to speak up, share this perspective with the school board and at the Strategic Plan community meetings that are coming up.
Thanks!
p.s. Linh-Co is right -- Common Core will be a whole new bucket of worms. For starters, it is potentially a costly, controversial and unfunded mandate. And yes, it will mean more testing.
Speaking of which, anyone have any info on the status or nature of the Common Core test pilots that are supposed to be happening at APP/Lincoln and John Stanford Intl Elem. this year?
Did that come up in the Assessments Taskforce meetings?
http://blogs.edweek.org/edweek/curriculum/2013/03/assessment_consortium_releases_2.html
New tests being designed for students in nearly half the states in the country will take eight to 10 hours, depending on grade level, and schools will have a testing window of up to 20 days to administer them, according to guidance released today.
The new information comes from the Partnership for Assessment of Readiness for College and Careers, or PARCC, one of the two big groups of states that are building tests in mathematics and English/language arts for the common standards. It answers one of the big, dangling questions that's attended the process of making these new tests: Given their promises to measure students' skills in a deeper, more nuanced way, partly through the use of extended performance tasks, just how long will these tests take?
The other group of states designing tests, the Smarter Balanced Assessment Consortium, has already come out with time estimates for its tests, as we reported to you in December. Those testing times—seven to eight and a half hours—are what students in 24 states will face; with PARCC's estimates out now, we have a better sense of what students in the remaining 22 states and the District of Columbia can anticipate.
Really? We're going back to "Bloom's Taxonomy"? It is a sort of simplistic pyramid of "learning" invented in the 50s (or 60's). The reality is, there is no evidence that this is actually the model of learning or knowledge acquisition. That whole thing was based on somebody's opinion, and nothing more.
Really? More tests for "synthesis" and blah blah blah? Just what we need. Replace "bubbles" with really subjective gobblety-gook. More state tests with uneducated test evaluators deciding whether your kid graduates - based on what? Somebody's opinion of whether he/she "synthesized" information in the standardized way.
Really? More lamenting that "foreign countries" are beating the pants off us? I thought that idea was also discredited. Other nations are "high performing" - because they don't educate everyeone. So, why are we still trotting that pony out?
This sort of "expert" is exactly why the professional status of educators is waning. Please return to Stanford.
I've got an idea. How about LESS ASSESMENTS? It's ridiculous the amount of assesment our students are now subjected to. Then we won't need to pay any experts to fly in from Stanford (or to just talk on the phone). How about we use what we've got if it works for teachers, at a price we can afford, or we scrap it.
-reader
Is the answer to focus less on assessment and more on content - for example, Core Knowledge? Won't the results follow? This is what New York city found with their pilot of Core Knowledge.
Does anyone out there actually work with kids with marginal math skills, day to day, other than Linh-Co and me?
Who cares about MAP-FRAPPE-CRAPPE tests when the basic skills shortages are NOT addressed, just passed along from grade to grade, with more Blame-The-Teacher-Higher-Order-Thinking-Reformie-Crap PD?
Since too many involved in this debate live in the leafy 'hoods of the Queen Annes and Wallingfords, and since too many are too far removed from the REAL economy of disposable employees working disposable jobs with disposable "health" care and disposable "retirements", no wonder the debate keeps focusing on How-We're-All-Gonna-Be-Einstein Idiocy. It is TOOOOO much work to get down into the basics of ANY social problem and figure out the details of making stuff work - let's hire MORE powerpoint parasites with big words, big sentences and big paragraphs and toss around Big Ideas!
The REAL beauty of all this freaking Jabborwocky is that the millions of kids with sound basic skills go off into the College Industrial Complex and turn into Powerpoint Parasites!
This nation, from lots of reasons readers of this blog should be familiar with, this nation has more wealth than ever seen in history. Do we export the best health care systems created and managed and run by the best people possible in health care? ha ha - I'm not answering that, because I don't think anyone is stupid enough to think the answer is yes. We do export the most cutting edge housing industry? the best and the most secure retirement systems? great elder care systems?
We export guns and CDOs and ponzi schemes, and corruption which would make Marc Antony & Julius drool with envy.
I've only skimmed Melissa's summary about Linda's Phantasy Land - What the hell is Linda Stanford talking about, other than her little San Jose Palo Alto bubble? She ain't talking about
ThoseFlyoverStates.
Reader, read up on PISA scores. The one study to support your point came from Stanford and the Economic Policy Institute. The study points out how US scores are affected by the disproportionate number of disadvantaged students compared to other industrialized countries. So once researchers control for that, US PISA ranking improved markedly. However, while you can control for things on paper, the reality is those kids are in our school system now. And they are not doing well.
off track
Assessments are going to happen.
What is it that people want?
But here is what I think.
The "job" of kids is to learn. Because we have concluded that they won't learn enough stuff, or the right stuff, on their own, people (either their parents if they are homeschooled) or teachers take on the task of "teaching" them. And right there, the perfectly spinning disk starts to wobble, because immediately, people (at least some of them) start to worry about (and measure, and evaluate, and try to figure out the importance of) what constitutes "good teaching" rather than what constitutes "good learning."
In a world where we trusted teachers more (and where we -- as a society -- believed that we hired people who were really smart, really understood their subject matter (in the upper grades) and really understood how to guide learning (whether by direct instruction, discovery methods, Montessori methods, whatever), we would need to impose (and hopefully do) very little assessment. It would rightfully be assumed that teachers did whatever level of assessment they needed to do to facilitate the learning of the kids in their care. You would test only at "barrier" or "entry" points -- or for things where there was scarcity (like seats in Ivy League Schools, or med school).
I miss the old days (pre-WASL) when it seemed to me like my kid took only one big test, the ITBS? -- in 3rd or 4th grade. Other than that, how his teachers assessed him seemed to me to be pretty much up to them.
The problem with bubble tests and other multiple choice MAP-ish things is that they are largely a waste of time (so either don't do them, or minimize them -- and when you do them, be prepared to think intelligently about how and whether the test results actually represent a child's learning. But at least they are (or should be) cheap -- though MAP seems to have figured out how to make them cost a lot) and fairly short (in terms of time spent) -- again, because MAP requires special set ups with computers and in taken multiple times a year, they have managed to take the two benefits of bubble/multiple choice tests and largely destroy them.
The problem with essay tests is that the scorers are horrible, so the test results are even worse. Back in the WASL days, I was horrified when I read the test questions (so vague and "eduspeakish" that I couldn't figure out what they were asking -- and I have 19 years of formal schooling!
And the answers? The way they were scored? It was dreadful. It was no surprise to my that my math-fluent SPED kid failed the WASL, and aced the HSPE a year later (and during that year, took almost no math -- so was going on what had been learned and retained from a year before).
In colleges (good ones anyway), teachers (not hired scorers in a warehouse a thousane miles away) grade student exams and papers. If this is what we want (longer essays and papers), we need to invest in enough teachers in at least a few classes so that teachers who know the students can read and assess student responses. Otherwise, assessments will just become longer, more expensive, and even less accurate (which was the case with the WASL).
I hope with the new common core (which I don't like and wish we hadn't adopted) tests, I hope that they do some rigorous assessment of the time, cost, and usefulness of the tests before they take the pilot nation-wide.
And, in fairness, I have been impressed (and reminded not to be so crabby), by insightful posts by parents who have found its scores useful -- whether in identifying high aptitude in a child where teachers (and maybe even parents) had missed it, or for those kids whose scores don't bounce around a lot, and whose parents seem to get a degree of comfort in data that they otherwise would not have.
But none of these cases justifies the massive time sink, facilities disruption, or cost of the MAP as we currently use it (because of course, what we really use it for is teacher evaluation, not assessment of kids). Assuming they want to keep it, I think we should take a critical look at its worse uses, and at least prune those. For example, its unreliability in K-2 suggests to me that we should drop it -- and instead have teachers use classroom assessments that are shorter, cheaper, geared more to the skills of each student (so you test knowledge, not whether they can use a mouse, or sit still). In 9th grade, it is of virtually no use, and is swiftly being overtaken by PSATs the following year, EOCs, etc. I had to grind my teeth when Danny Westneat came out with a pro-MAP article, saying we should keep it because it helped identify gifted kids. Well, it isn't doing that in 9th grade, and we have no program to move them into at that point anyway, as APP entry closes at 8th grade, and all classes that APP kids take are available to other kids in any case.
If we got it down to 3rd through 8th grade (or maybe 3rd through 7th -- if the 8th grade scores have the same issues as the 9th grade ones), limited it to those families who wanted it (ample opportunity to opt out), and moved the testing to some spot (like John Stanford Center) where kids went THERE to take the tests -- rather than having it all done in each building -- it should cost significantly less (fewer students involved), impact school facilities much less (because it is administered downtown), and free up everyone else's time to get back to learning.
I'm confident of one thing: Common Core testing is an unfunded mandate will get swcrewed up.
In terms of taking this test out of class time, there is NO way anyone is going to fund additional transportation, proctors etc. Our kids will just have to suffer the loss of educational time- not to mention exposing them to ridiculous amounts of tests and risking test fatigue.
I looked at a private high school. This type of testing didn't exist, and it was refreshing.
The time to administer the SBAC tests is nearly identical to the test necessary to administer the current MSP, HSPE, and EOC assessments.
Also, Linda Darling-Hammond is NOT on their board. She is a member of their Technical Advisory Committee, though. The TAC does not have a governance role in the consortium.
I'm not interested in debating the pros and cons of the SBAC tests. I simply wanted to provide some facts for others to consider.
--- someone who knows
--- someone who knows
--- someone who knows