Signed, Annoyed
A thread was requested (please note I didn't write this thread but I believe the info is accurate). Weigh in on the MAP testing for use in Advanced Learning placement. (They used to use the WASL.) I think on the one hand most of the money for Advanced Learning goes to testing. It's just ridiculous that more of the money can't go to programming. However, using MAP may benefit those students who can use computers well.
From Annoyed's comments:
The change this year was that MAP testing would determine if students were qualified to take the CogAT, rather than the reverse, where students first take the CogAT, then if they have qualifying scores, they take an achievement test for math and reading.
In recent years, I believe the K-2 kids took the the Woodcock-Johnson test and the older kids took the ITBS test for reading and math achievement.
Now the MAP test is being used as a gatekeeper and preventing many students from taking the CogAT. There may be some justification for this in the higher grades, if MAP scores show a strong correlation to ITBS scores, but I have not seen evidence that MAP in the primary grades is a valid measure of a student's academic performance when testing for highly capable programs.
From Annoyed's comments:
The change this year was that MAP testing would determine if students were qualified to take the CogAT, rather than the reverse, where students first take the CogAT, then if they have qualifying scores, they take an achievement test for math and reading.
In recent years, I believe the K-2 kids took the the Woodcock-Johnson test and the older kids took the ITBS test for reading and math achievement.
Now the MAP test is being used as a gatekeeper and preventing many students from taking the CogAT. There may be some justification for this in the higher grades, if MAP scores show a strong correlation to ITBS scores, but I have not seen evidence that MAP in the primary grades is a valid measure of a student's academic performance when testing for highly capable programs.
Comments
While the collection of changes is diverse, they all have one thing in common: they will all lead to further disproportionate under-representation by students from low-income homes.
The use of the MAP test as a gatekeeper is one. Because it is a computer-based test, it heavily favors students, particularly in the early grades, who have and use a computer at home and puts a heavy bias against students, particularly in the early grades, who do not.
The District has eliminated Spectrum Young Scholars, a program specifically designed to find and nuture gifted students who have been privileged with exposure to academic opportunities.
The District no longer allows students to qualify for middle school Spectrum in either math or reading. Now, to get into the Spectrum LA/SS class, a student must demonstrate high achievement in math.
The District has eliminated middle school A.L.O.s. Isn't it odd that we recognize non-Spectrum and non-APP students working beyond grade level in elementary school and provide them with access to more challenging and rigorous academic opportunity there, but suddenly, that is no longer necessary when the students reach middle school?
And, on a continuing basis, the District absolutely refuses to offer any quality assurance or even quality assessment for advanced learning programs so that low-income students can be locked into programs that exist only on paper but not in real life.
I agree with Charlie that this is likely to strongly - and adversely - decrease the participation in advanced learning of children from low income homes, as well as children whose home language is not English. This is really, really not okay.
I wrote Bob Vaughan to ask why this decision was made, and it appears to be entirely because there was a significant surge in applications for testing this year. This year there were 5,100 applications, up from a long-standing usual of 2,300-3,000. I didn't think to ask if there was an increase last year, as well.
I plan on asking some more questions, but this appears to be an unfortunate reactive decision on the district's part. I would guess that the surge in testing has a lot to do with issues and uncertainty created by the New Student Assignment Plan, and applications will, as the plan (hopefully) stabilizes and improves, drop back to more normal levels, at least in proportion to enrollment.
Other districts do not appear to use MAP as a gatekeeper to advanced learning testing.
I don't know how much it costs to test each kid. I do know, based on a letter APP families received from SPS, that for every APP kid who rides the bus, the district gets about $3500 from the state (compared to $350 for regular education students).
I realize that the testing/bus revenue issues are, in fact, separate issues, but it's hard to ignore that on the one hand, the district is asking APP families to help them out, while on the other, SPS is looking at further chopping up that very program, this time at the high school level, and reducing access to all advanced learning programs.
If MAP is used as the cutoff, without knowing the CogAT scores, has the District really fulfilled the intent of the law? The burden is now on families to provide the additional test data (at great cost) and go through the appeals process.
But if you opted your child out of the MAP, the District would need to administer the CogAT, since there would be no achievement data available, yes?
The draft report on District MAP data shows very spotty testing for K students. Where is the data that supports using MAP as a qualifier for this age group? I don't find it acceptable to tell families to just wait another year. Are the students truly not capable or is the test not valid for this use and this age group?
Is it suitable for use as a placement test?
It seems to me like a big argument against using MAP for advanced learning is that parents who don't know the system (like me), won't even have any idea that they should apply for their child in October (before they even see those MAP scores).
Not that it really matters in my case, since she's at a school with no Spectrum, no ALO. But I would have liked to be made aware of the possibility for her.
There are three routes to applying for Advanced Learning; a teacher alerting you to the program believing it may be good for your child, seeing information at your school (bulletin board or information in the office) or hearing about it at a PTA meeting or in the parent newsletter.
But maybe it might be worth finding out if the district, in its information for kindergarteners and/or new students, has info on Advanced Learning. I mean no one can hold your hand and feed you all the information but if it isn't available in district info handed out or at your school, then something is wrong.
As we later learned, there was open hostility toward the program from many parents, teachers, and even the principal. Several teachers tended not to recommend students for testing and one parent was actually discouraged from doing the testing, by a teacher. If there was notification, I certainly missed it. Perhaps it was in small print somewhere in a newsletter.
So I can empathize with Missed the Deadline and wouldn't assume it was a matter of needing hand-holding.
Unfortunately, there is no option to appeal if you missed the deadline and didn't apply.
-Been there
Poor reliability of K-2 MAP results means that they should NEVER be used for a screen for APP and ALO.
And/or all children who score high enough on the MAP need to be recommended for testing automatically. If MAP is a gateway it should usher children in more than keep them out!
They all want you in the still-public system. If you take your little guy private, it hurts their budget. Even if you have no intention of leaving the system now, you may find that a typical classroom is not going to meet your child's needs.
This is another example of how the supt is actively changing the structure of our options for education in this city. It should signal a letter writing campaign for anyone reading about your situation - even if it is "hearsay" (for us blog readers), it is something the people at the top should hear about,
My point is that having 5100 applications this year may partly reflect the fact that the district increased AL opportunities. Parents at Bryant may not have tested in the past because they were happy at the school and wouldn't have moved to a Spectrum school or Lowell anyway, so why test? But now, there is something "new" called at ALO, and if you want your child in the "new" thing, you may have to test. Voila, more applications. It would be interesting to know where the surge in appls is coming from.
Unfortunately, since schools are compared in part on standardized test scores, there is a strong incentive for parents, teachers, and principals to oppose sending their best scoring students off to some specialized program at a different school, even if it is the best thing for the student.
In general, the idea that the school's primary goal should be maximizing the educational opportunities and pace of learning of all students seems to be lost in this district.
I ran into SEVERAL parents from our school (North Beach) with older kids. They received letters from the district encouraging them to go to testing based on last Spring's WASL.
Again, I am new here, and I agree with most everything on this blog, but if the above is true, the district is making efforts to test more kids than the posters here would have you believe.
Now, we'll just have to see what happens with the results of the testing.
Interesting how concerned they are that doing anything that might " seriously damage the district's communication strategy on MAP, and buy-in from teachers. Got to keep up the appearance of informing instruction and all that.
MAP can do anything!
So while there are clearly downsides to the MAP approach for some families (including for my kindergartner), there are upsides for other families.
Also, if applications for APP testing are up so dramatically, we might also consider that a positive endorsement by parents of the program -- not just a negative reaction against the new student assignment plan.
My daughter is new to APP this year, and many, many parents w/children in private school tell me they wish they weren't paying $25k/year for academics that most people don't seem to think are any better than APP.
She said to call back in a week if we haven't heard anything.
What both versions of this memo show is a broad brush attempt at using an inferior product for purposes for which it was never intended. Inferior because it was never REALLY evaluated effectively against competitors. Is using MAP in this way a new cost-saving measure? LOL! oh gawd, I'm killing myself.
It also shows SPS talking outa both ends:
Let's put forth a message that MAP is good for our kids (inform instruction), while going down a different path altogether (evaluate teacher "effectiveness," SpecEd eligibility, summer school, math courses)!
Ironic in light of MGJ's ludicrous statement at the Board meeting.
"Assessing every student and placing them according to ability sounds like tracking and discrimination…and we’ve moved light years away from that."
Right, MAP is color-blind, mind-blind, curriculum-blind, the list goes on.
P.S. my last post was to Jessica at 11:18
Jessica de Barros states that if MAP were to be used for purposes other than "informing instruction", then "more than one point of data - preferably 3 points - [should] be used when drawing conclusions about student performance."
For use in Advanced Learning, the permission was given to use MAP for the purposes of identifying students from under-represented groups. The assumption is that "MAP would be used along with other student performance data points."
In bold, it states "at least two other types of data must be available to support the decision-making process."
So, yes, multiple points are being considered for those that have other data - WASL, etc. - when recommending students for testing. Yet, the 85% MAP cut-off for additional testing is eliminating access to multiple data points when being considered for Advanced Learning opportunities. Which makes it a high-stakes test.
What about a K student that has no other data points besides MAP? Are they out of luck? Notice that the other uses include 6th and 9th grade math placement.
I'd love to see this information. Don't forget, the district is planning on spending tens of millions of dollars on this system. I might add-these dollars will be taken out of classrooms.
Worth analyzing.
Public school has its complications. Lowell APP seems to be oversubscribed. TM APP seems to be under-subscribed in grade 1 but well-attended in older grades. It's definitely a quandary for everyone when "success" in APP enrollment puts pressure on budgets and schools' capacities.
RPM i'm trying not to take offense at "keep it real here".. an unproven (other than not ready for statistical analysis in grades K-2) test is being used as gatekeeper to adv learning for Kindergarteners. the exam has little to do with propensity to learn, and don't even get me started on how 1 teacher can manage to proctor 30 kids in a computer lab where the kids can't even keep their headphones on staight and the district expects to have accurate reflection of knowledge in scores.
yes, it's wonderful the district is taking MAP scores and letting parents know perhaps they should test their child, but it shouldn't be used the other way around as a gatekeeper.
I hope this analogy isn't too strained, but a good screening test is used for "asymptomatic" people, so in this case, you can screen for potential giftedness even when there are no "symptoms" or clues that have been detected in other ways (ie, no application filed). Great, I support that, especially if the goal is to increase diversity and rely less on self-selection for testing.
However, once someone has "symptoms," you can't just rely on the screening test and ignore other data.
Imagine if a women went to her doctor with a painful lump in her breast and the doctor said "Well, I'm not going to order a mammogram until I run another test that has never been shown to be effective at detecting breast cancer. How about we do a colonoscopy and if that's positive, well then, maybe you can get a mammogram?"
This is absurd, right? But it's exactly what SPS is doing this year. MAP is not a validated tool for identifying giftedness in young children (right? can anyone refute that with evidence?). Families who submitted the application packet provided evidence of potential giftedness but the district won't apply the appropriate "diagnostic test" (CogAT) unless the kid gets past the arbitrary screening test first.
And for those who say, hey, you can try again next year, you have probably not lived with a miserable, bored, and unchallenged child for a year. It's not fun for anyone. I am very sensitive to this issue because I've seen the transformative power of a good program fit in my own family.
Strangely his MAP percentile ranking fell almost 20 points from Spring 2010 to Fall 2010, so he would not have qualified for the Cogat testing in any case.
My 3rd grader (who did not receive a letter) scored in the 95th percentile for reading and the 84th percentile for Math. He scored the exact same way for Fall 2009 and then his scores fell for Winter and Spring cycles. I think he's a bright boy, but not a candidate for advanced learning.
I think too much weight is being given to the MAP scores and to scores for one specific testing cycle. I think parents and teachers probably know best which students are the best candidates for advanced learning no matter what test scores show.
No. No they don't.
Well, I suppose if you are willing to take what they are offering with a smile and a hearty "Thank you sir, may I have another", they may want you.
But, threaten to go private and all you'll hear is, "Don't let the door hit you in the arse on the way out", followed by the chirp of crickets. If you actually go private, don't expect anyone to call and ask what caused you to leave SPS in general or the APP program in particular.
MG-J has stated before that she doesn't see private schools as competition. SPS does not see families as clients or customers that need to be wooed. They don't care if you find a better product or service because there are many more who don't have that choice.
Unless things have changed, testing is not required to be in an ALO. My understanding is that in theory it is more of an opt-in program. So, that shouldn't drive that many more families to testing - if anything it should be fewer, since they might get their needs met locally. At both Lowell and Thurgood Marshall last year, the non-APP classes were called ALO, but to my knowledge, the ALO classes weren't any different from any other Gen. Ed. class.
If you dig through the blog you'll find plenty of posts that lament the lack of standardization among Spectrum/ALO and call out many schools for being "Spectrum/ALO in name only". One of the attractions of APP was that in being self-contained at single sites, there wasn't ever any sense that one APP school was better or somehow "more" APP than another. Of course, that foundation has been eroded, as well.
I wish that were true at my child's school.
The kids considered for the ALO pull outs at my child's school must be District identified as Spectrum or APP. It is important for all to understand that ALO comes with little oversight (schools must submit a plan for ALO but the District doesn't have any guidelines about what the program must/should have) AND the program, once certified (what a joke!) by the District comes with no money. No money to pay for pull outs (if that is the model) or any materials, etc. Thus, our school limits ALO opportunities to APP or Spectrum kids who have chosen to remain at the neighborhood school rather than move for a program.
In theory and in its origination, ALO was to be an opt-in program. In practice, I understand from talking to other parents that my child's school is not the only one that limits access to it.
That's disappointing to learn. If I were suspicious; I might suspect that MG-J/SPS were trying to bring about the slow demise of APP/Spectrum.
Now we have to test into ALO? That surely indicates that an ALO school is as rigorous as Spectrum/APP. Before long they can claim that every school can meet the needs of highly gifted students and disband APP/Spectrum altogether.
Another nail in the coffin *sigh*
The Advanced Learning site says that ALO students must be either district-identified or teacher-identified -- there doesn't seem to be a full opt-in position. It sounds as though depending on the attitude of the teachers/administration at your school, you may not be able to qualify except via formal test scores.
Helen Schinske
Another Parent
Here is a quotation form the NWEA technical manual.
[page 54] "It is generally known that the assessment of second graders is inconsistent for many reasons including the mixed reactions that these students have to taking multiple-choice test items. For these resaons, NWEA test users should expect lest consistency in the test scores of second-graders."
Technical Manual for the NWEA MAP and ALT Assessment (NWEA, 2003).
I received this manual by email from Jessica DeBarros. I was not able to find a copy of it on-line, so I cannot provide a link to it.
It must be that the consistency of test scores for K and G1 is even worse than for second graders. From this technical manaul we cannot learn just how bad are the statistics are for these two lower grades, because the statistics for K-1 are not reported in this technical manual.
It is probable that a high proportion of K-1-2 children did not obtain statistically valid MAP test scores.
A student's test is not valid if the SEM (standard error of measurement) for a particular test event falls outside the range of 1.0 and 5.5.
I wonder how many children did not get flagged for CogAT testing because their SEMs fell outside this range?
http://www.scribd.com/doc/41812304/NWEA-Technical-Manual-2003
That may be true for individual schools (I just don't know), but I have a hard time believing that all of the non-APP students at Lowell and T. Marshall last year tested into Spectrum or APP. Unless I am mistaken, there was no Gen. Ed. at either of those schools - just APP and ALO. Supposedly, Lowell got a chunk of T.T. Minor's students and T.Marshall kept a chunk of it's previous students (although I'm sure there was a lot of churn).
Maybe last year was a transition year and they started testing everyone this year? If not it just points to the variability in what can be labeled ALO.
I think you are mistaken. There is APP and a general ed program where children may participate in ALO at these schools. In SPS it's never general ed or ALO. It's always general ed and ALO. It's the same at every school.
Just because you have an ALO program doesn't mean that all the students participate. Likely very few participate.
Lowell's website says
ALOs serve two primary student groups: (a) district-identified students who are academically highly gifted/gifted, and (b) teacher-identified students who demonstrate skills and readiness for participation in an accelerated and rigorous curriculum that is based on Spectrum curricular guidelines.
So it's both test based and teacher identified at Lowell. That's the district-wide expectation, I believe. Check the advanced learning page and you'll see the same definition.
In practice, however, I bet schools who say the program is only for district identified students are just getting push back from teachers. If the teachers don't want to identify any additional students, effectively it's only offered for kids who test in.
If schools are doing something else, that is just further evidence of failure by the Advanced Learning office.