Performance Evaluations
With all of this talk about Performance Management I thought it would be a good time to review the Performance of the Board Directors and the Board as a whole. I know that the Board does their own self-assessment, but I can't find it. Besides, it is impossible for anyone to hold themselves accountable. I simply have no faith in self-policing.
For accountability purposes we need some objectively measurable outcomes for the Board job.
The Board job, as I have often written, has three components.
First is to serve as the elected representatives of the public. This includes:
A. Representing the public's interest
B. Representing the public's perspective
C. Advocating for the public's perspective
D. Advocating for community engagement
E. Providing community engagement
Second, the Board is supposed to oversee the management of the District, to supervise the Superintendent. In that role they should:
A. Confirm that the Superintendent's decisions comply with state and federal law
B. Confirm that the Superintendent's decisions comply with Board Policy
C. Confirm that the Superintendent's decisions are based on data and sound rationale
Third, the Board is supposed to serve as a Policymaking body. In that role they should:
A. Direct the budget policy of the District
B. Direct the academic policy of the District
C. Make sound decisions regarding the District's property
D. Carefully review the recommendations brought before them for action and make their decisions based on data and sound rationale
Have I left out any elements of the job?
Next comes metrics, assessments and benchmarks.
For accountability purposes we need some objectively measurable outcomes for the Board job.
The Board job, as I have often written, has three components.
First is to serve as the elected representatives of the public. This includes:
A. Representing the public's interest
B. Representing the public's perspective
C. Advocating for the public's perspective
D. Advocating for community engagement
E. Providing community engagement
Second, the Board is supposed to oversee the management of the District, to supervise the Superintendent. In that role they should:
A. Confirm that the Superintendent's decisions comply with state and federal law
B. Confirm that the Superintendent's decisions comply with Board Policy
C. Confirm that the Superintendent's decisions are based on data and sound rationale
Third, the Board is supposed to serve as a Policymaking body. In that role they should:
A. Direct the budget policy of the District
B. Direct the academic policy of the District
C. Make sound decisions regarding the District's property
D. Carefully review the recommendations brought before them for action and make their decisions based on data and sound rationale
Have I left out any elements of the job?
Next comes metrics, assessments and benchmarks.
Comments
- there is a team of college credentialed, over 6 figure a year salaried consultants,
- there is at least 6 months of analysis,
- you come up with a powerpoint over 40 slides long using at least 100 terms from total quality management jargon ... oops! TQM was the 80's ...
At the end of today's Seattle Times article about Randy Dorn being pulled over for a possible DUI is this paragraph (bold emphasis mine):
The Eatonville resident based much of his election campaign two years ago on getting rid of the controversial Washington Assessment of Student Learning, saying it doesn't provide meaningful feedback and is too time-consuming. Starting this year, high school students are taking the High School Proficiency Exam instead of the WASL, while students in grades three through eight will take the Measurements of Student Progress.
SOURCE: http://seattletimes.nwsource.com/html/localnews/2011417649_dorn23m.html
And yet, here in Seattle, SPS is administering MAP to kids in K-2 as well. Is Seattle the only District in the state that's doing this?
Is MAP even designed for or appropriate for these grade levels?
If WASL wasn't administered to such young kids, why is it OK for MAP to be?
I still don't understand why 5-year-olds are being forced to take a standardized, computerized test three times a year in Seattle.
Many are still learning to read, don't necessarily know how to use a computer or mouse, and are losing out on valuable library time in order to take this test.
Again, whose idea was it to bring MAP to SPS, and why is it being given to kids so young and so often?
It all seems excessive, not very useful, costly ($4 million in the levy allocated to MAP, right?) and not the best use of our kids' time.
Testing in Kindergarten: The Realities and the Dangers
http://www.education.com/magazine/article/testing-kindergarten-realities-dangers/
by Anna Weinstein
Topics: What to Expect in Kindergarten, Standardized Test Preparation
Standardized testing in kindergarten. Perhaps unimaginable, but it is happening in some states. And for many educators and parents, this reality is cause for great concern.
“There is clear evidence that these types of tests are not appropriate for young children,” says Samuel Meisels, expert in childhood assessment and president of the Erikson Institute, a graduate school for child development, in Chicago. “In the first eight years of life, there’s a great deal of variability in how kids learn, and standardized tests tend to see children in a very homogeneous way,” Meisels says.
According to a recent report released by Alliance for Childhood, an advocacy group based in Maryland, research shows that standardized tests do not accurately measure young children’s knowledge and are not reliable indicators of future school success. Joan Almon, executive director of Alliance for Childhood, explains that young children simply do not have the skills to overcome their physical and emotional needs on any particular day. “Children are very subject to how they feel,” Almon says. “If they’re hungry, for instance, or sleepy or grumpy, their test scores will be affected.”
Meisels explains that this focus on standardized testing is a result of the structure of accountability set in motion by No Child Left Behind (NCLB), the largest high-stakes testing endeavor ever implemented on a national level. Though standardized testing through NCLB does not begin until grade 3, some states have employed the testing in earlier grades.
The belief of some superintendents and principals is that by beginning the testing earlier, children will be better prepared for the tests when they reach third grade. “But this isn’t based on any long-term research,” Almon says. “I’d be surprised to see that standardized tests in kindergarten would yield higher test results in grade 3.”
But, despite the concerns, test preparation is happening in kindergartens across the U.S. According to a report released by the Alliance for Childhood, teachers are spending an average of 20 to 30 minutes each day on activities associated with testing.
Meisels says Florida and Texas are two examples of states using standardized testing when students enter kindergarten in an attempt to measure the success of the prekindergarten programs. This approach, Meisels says, has a number of significant flaws.
The first flaw Meisels points to is that tests given in the fall of kindergarten serve solely as posttests, so there is no measurement built in for progress. “We don’t know where the kids started from,” Meisels says. “We don’t know whether they made a lot of progress or very little progress.”
Another problem with the approach is the lack of controls for learning loss over the summer months. “There’s a phenomenon of summer loss in education,” Meisels explains. “We know that children have extremely different summer experiences, and some kids come back in the fall and have lost a lot of achievement.” (continued)
Testing methods are another concern. Meisels explains that the tests are simplistic achievement tests of basic literacy and simple math, yet they aren’t truly representative of literacy or math skills. “They are just a few things that kids need to know,” Meisels says, “but it may be that some children have great strengths in other areas that aren’t tested at all.”
Another flaw to the approach is the effect tests can have on teachers and how this trickles down to students. According to Meisels, there is substantial evidence that teachers’ perceptions of young children can be influenced by test scores. If these test scores are not accurate depictions of what children can do, this can result in teachers having inappropriate negative impressions of the children, Meisels says. As might be expected, research also indicates that young children do not have the experience or built-in self-esteem to be able to withstand these negative perceptions over a period of time.
The Alliance report summarizes the position of the Association for Childhood Education International (ACEI), stating that “standardized testing in the early years causes stress, does not provide useful information, leads to harmful tracking and labeling of children, causes teaching to the test, and fails to set conditions for cooperative learning and problem-solving.”
So if not standardized tests, then what--how should teachers evaluate students’ progress? According to the Alliance report, research has shown the best way to “test” children in the early grades is through performance assessments or observational assessments. With these assessments, teachers observe, record, and then evaluate students by comparing their observations to explicit, well-developed standards of performance across the entire curriculum. The key is that children are unaware they are being tested.
With the pervasive use of standardized testing and the overall emphasis on performance and knowledge gain in kindergarten, parents should keep in mind that they can be proactive about their children’s education. If an assessment or test of any kind indicates that a child needs help, parents should feel comfortable talking with the teacher about the accuracy and validity of the test. “Believe in your child. That’s first,” Meisels says. “Before I got too excited, I would try to get the teachers to explain more fully, to get more evaluation if necessary, and I would want to get another opinion.”
Meisels emphasizes that it’s more important to believe in our children than to believe in the tests. At the same time, he says, we should remember that the goal of both the parent and the teacher is to help the child. “I always try to counsel parents to work in a partnership with teachers,” Meisels says. “We’re all trying to help the child to have success in school.”
Anna Weinstein is a freelance education and academic writer. She has written and edited textbooks and materials for many educational publishers, including McGraw-Hill, Harcourt, Houghton Mifflin, and Rosen Publishing.
Did the SPS School Board approve MAP the tests?
If so, do they know whether they are worthwhile or worth the cost?
Do our School Board members all believe that such testing is appropriate for such young kids?
Did the School Board know that Seattle School Superintendent Maria Goodloe-Johnson is on the Board of Directors of the company that manufactures the MAP test and sells it to SPS? (Northwest Evaluation Association: http://www.nwea.org/about-nwea/our-leadership)
Doesn't this potential conflict of interest concern them?
If the Board does not have answers to all these questions, then why aren't they asking and getting them?
The district has been doing various kinds of tests in K-2 for as long as I've paid any attention (and in this district as a child I got a bunch of testing in those years as well -- maybe not in K, but certainly in 1 and 2). So while it may indeed be an issue of concern for some, it's not really a *new* issue.
Helen Schinske
But I still don't understand why SPS is administering standardized computerized tests to kids below Grade 3.
Also, that means that SPS kids in Grades 3-8 are having to take the MAP and the MSP tests. Seems like a lot of testing.
Helen -- have the past tests that you refer to been on computers and given to kindergarteners as well? What were they used for? Were they given three times a year?
I just don't remember this obsession with testing in past years.
They just need something to obsess over..... lest they have to look at the overall ongoing mess.
According to the information on Source, "The Developmental Reading Assessment (DRA) is a state mandated reading test for all second grade students to be administered each fall. ... In the spring, SPS requires all K, 1 and 2 grades to use DRA and the follow-up analysis and parent letters for all 3 grade levels will be provided. Spring score input is available for grades K, 1 and 2. (Using the DRA for grade levels other than specified is always an option, but reports and parent letters will not be available for them - and score input on the Source will not be available for the optional grade levels.)"
I don't really see what the big deal is about the MAP being on a computer. Seems if anything much more child-friendly than a scantron form.
Helen Schinske
I would be VERY surprised if the DRA does not have higher reliability and smaller standard deviations than MAP.
The DRA probably is considered a standardized assesment. It is criterion-referenced rather than norm referenced. Criterion-referencing is superior to norm-referencing. The NAEP - the gold standard in student assessment - is criterion referenced, and has many questions that require written answers. Thus is is not administered by computer. It is graded by well-trained people, so as to minimize the variance in scoring by different graders.
I don't know whether MAP is criterion or norm-referenced.
Gavroche asked "Did the SPS School Board approve MAP the tests?"
I saw in board minutes from last year (don't remember where) that the Board approved DRA being dropped and replaced by NWEA's MAP.
I was disappointed to learn this.
If Helen is right that the DRA is required by the State, then it seems that the Board has made a decision that violates state law.
Gavroche asks: "If so, do they know whether they [that MAP] are worthwhile or worth the cost?"
I don't recall seeing any cost-effectiveness analysis in the Board minutes.
MAP is more compatible with high stakes testing, which calls for quantitative comparable scores. I guess it is costly or not even feasible to translate DRA assessments into quantitative, intercomparable performance measurements that can then be used to support the making of important decisions about students, teachers, and schools.
Gavroche asks: "Do our School Board members all believe that such testing is appropriate for such young kids?"
Of course, if you ask any board member their belief, you will find that their belief accords with their vote, regardless of what the best available peer-reveiwed research says, and regardless of any consistent problems detected by teachers, parents, and students, and regardless of consistent complaints from andy of these quarters.
The research is quite clear on this point: Standardized test scores lose their meaning, value, and validity when the tests are used within a high stakes testing content.
High stakes testing is exactly how this District intends to use assessments.
The Board will vote to approve the revised C40 and the repeal of C41 in April 7, despite the fact that they have they have received emails from experts that confirm that these are blatant high stakes testing policies, have been sent a URL link to a very recent, peer-reviewed National Academies of Sciences expert panel literature synthesis report that makes an unqualified assertion that high stakes testing is bad practice.
It looks like the only way to induce SPS to abandon high stakes testing is to go to court. But can we raise the money for the legal costs? Will SPS appeal all the way to the U.S. Supreme Court? The current court may well grant SPS a win. Will the parents be able to garner the resources necessary to fight every appeal by SPS?
Then, to follow state law, the second grade teacher had to use the DRA again. On these same kids who already showed that the results were meaningless for them. Another 14 hours of teaching time gone.
Plus, I have heard that in some schools, PARENT VOLUNTEERS administer the DRA, just so the teacher doesn't lose 14 hours of teaching time. This is hearsay, so would need some first hand corroboration.
A teacher I met at a party, who taught at a struggling school, audiotaped all of her kids' DRA assessments so she could go home and listen carefully to score them as accurately as possible. How many teachers did that?
And then there's my son's experience with the DWA. Oh, the only time in APP where the teacher taught to the test. Curriculum night, his third grade teacher confidently said all the kids would do well on the DWA. She managed that by giving 6 writing assignments, one for each strand assessed. Other than that she gave NO writing instruction or assignments or nada. And of course since the DWA was given in February, there was zippo writing instruction or practice from then on.
What I remember about the DRA is that the evaluator maps where the student is on a comprehensive range of criteria. The parent gets really clear information about what skills the student has mastered, and what are the next steps that that the student is working on. I would think that this assessment would provide a teacher with very useful formative assessment, unless the student has topped out.
I don't see why the teacher can't just pull the DRA for the next grade level when they find a studnet is topping out in the current grade level's DRA.
I don't think it is ideal to have parents to administer the DRA. The value of a human administered assessment is that the evaluator can help to see that the student's score is less affected by mood and other confounding variables.
There are very good reasons why the best IQ tests are administered by highly trained Ph.D's, rather than by machines!
On a separate matter:
Why don't you ask your son's teacher to implement Reader's and Writer's workshop? I think it is brilliant, though I have heard criticism that it is weak on grammar. I feel grammar is quite important, and am dismayed that SPS might not being providing a good grammar curriculum.
I understand on the other hand, that there is a good rationale for separating the writing-for-expression curriculum from the writing mechanics (grammar, punctuation) curriculum, at least in elementary school. By high school or perhaps by middle school, I would expect that students should be able to integrate their communicative writing skill with their writing mechanics knowledge.
No single assessment will be ideal for every possible application and situation. So I am not surprised to hear criticisms of the DRA & DWA such as Dorothy has given.
IF the MAP is overall a much better instrument for providing diagnostic, formative, and summative assessment for parents and for teacher's benefit, then I am in favor of the MAP replacing the DRA.
I highly doubt, however, that the relative value of the two instruments (MAP vs DRA/DWA) for helping a teacher to improve instruction to a specific group of students and to individuals in the group had anything to do with the rationale for having MAP replace the latter.
As I said before, I think the reasoning had largely to do with how amenable the test results are for a high stakes testing regime.
How many times do you hear them say "My constituents want..." or "The Public demands..." They should say it at least twice per Board meeting. How often do they actually say it? They never do.
How many times do Board Directors fight for things that the public wants? They should do it at least three times a year. How often do they actually do it? They never do.
How many times has the Board insisted that the Superintendent conduct authentic community engagement? They should do it at least once at every Board meeting. How often do they actually do it? They never do.
How often does the Board respond to community input, such as respond to testimony at Board meetings or reply to emails? They should do it thirty times a month. How often do they actually do it? They almost never do.
How often does the Board insist that the Superintendent comply with Board policy? They always should, but they never do. How often does the Board hold the Superintedent accountable for violating policy? They always should, but they never do. How often should the Board insist on rationale supported by data and logic for the Superintendent's decisions? They always should, but they never do.
How well does the Board review the motions brought before them for approval? They should always seek an independent perspective but they never do. They should always review the source documents but they never do. They should always apply critical reasoning skills but they never do.
The DRA goes across grade levels, but I believe the version used in Seattle elementary schools goes only through level 44, which is about grade 5. Some schools do cap the level at which they will stop assessing in a certain grade, but Seattle schools do not, as far as I am aware (my son got a 40 in kindergarten, according to what it says on Source).
I don't remember getting any detailed report (though it's six years ago, so maybe I just don't remember) -- just subscores for accuracy/fluency/phrasing/retelling.
Helen Schinske
That's kind of like saying that a chest X-ray is superior to a colonoscopy, or a ruler superior to a scale. It depends what you are trying to measure and what you want to do with the results.
Helen Schinske
URL: http://seattletimes.nwsource.com/html/opinion/2011420829_guest24bailey.html
Guest columnist
Teacher seniority pays off, especially for students
Teacher seniority helps ensure quality education for students, writes guest columnist Patricia Bailey. The Seattle School Board should not try to undo seniority in upcoming negotiations, as some community petitions are urging.
By Patricia Bailey
Special to The Times
WITH community petitions calling for the Seattle School Board to undo teacher seniority in the upcoming negotiations, it's time to put an end to the myth about ineffective public-school teachers being protected from termination. Those of us who teach in the K-12 public schools use seniority, like all union and governmental jobs, to determine the order of layoffs, but seniority is not the same as tenure.
We do not have the lifetime job security afforded a tenured judge or professor. Unlike them, we can be dismissed for doing an ineffective job. In fact, the recent district audit by McKinsey & Co. noted the district's underutilization of the job-termination mechanisms already in place in the teacher contract. In Seattle, student performance has long been a part of teacher evaluations and can factor into an unsatisfactory evaluation.
If principals are doing their state-mandated duty to evaluate teacher performance, how could there be unsatisfactory teachers in the classroom? During a Seattle teacher's first two years of employment, he or she can be dismissed without probable cause. This is the time for principals to evaluate and counsel out those new, but unsuited, to the profession.
After two years, teachers are afforded due process before termination, but this does not prevent a principal from evaluating someone as "unsatisfactory" and pursuing dismissal. After giving the teacher a chance to improve, employment can be terminated. It is as simple as that.
Using seniority to determine the order of layoffs is the only fair way to decide between who should lose his or her job when two workers are both evaluated "satisfactory." Seniority eliminates any possibility of layoffs being capricious, arbitrary or discriminatory, or from using layoffs for retaliation or domination purposes. It also encourages the retention of experienced teachers.
High-quality teaching comes with experience. Studies show it takes about five years for teachers to reach optimum knowledge and skills and become truly proficient in the classroom. There is no denying the exuberance of a new teacher can be a delightful addition to a school, but this should not be confused with high-quality teaching, which occurs in classrooms featuring perhaps less fanfare and more-refined techniques.
If a district wants high-quality staffs, then it should protect those teachers with five or more years. It should also recognize that teachers of four years have more professional growth under their belt and are closer to reaching their optimal teaching years than a first-year teacher. Consequently, the "last hired is the first fired" ensures the best professional services for our students.
But perhaps the most compelling reason to keep seniority is as a voice for student health and academic achievement. Generally speaking, it is senior teachers who speak out on controversial decisions including school closures, textbook adoptions, increases in class size and other student-impacting issues. If teachers feel their jobs are threatened, I fear these voices would be silenced.
Contrary to urban lore, the teacher's union cannot prevent terminations if the district administration has done its job properly. In Seattle, student performance is already part of teacher evaluations, so when organizations call for the elimination of seniority or the use of high-stakes state test scores in determining layoffs, they are misguided.
Although it might financially benefit a district's budget to keep inexperienced teachers over their seasoned counterparts, it is sheer folly to suggest it could lead to better student outcomes. As the saying goes, "a new broom may be stiff, but an old one knows the corners."
Seniority comes with a cost, but for the student, it is priceless.
Patricia (Pat) Bailey, a veteran Seattle teacher, is a former executive board director of the Seattle Education Association.
http://www.seattleschools.org/area/
acastan/stan/read/dev_stg_read.xml
I thought this was the most informative part of the district's grade report form.
Re criterion vs. norm-referenced standardized tests: criterion-referenced tests are basically pass/fail tests that assess whether a student meets a certain standard. A driving test is a good example. The scoring can be more complicated than pass/fail, as with the WASL, but beating the cut score is generally regarded as very important.
Norm-referenced tests compare the student to a large population of other students, to show how unusual the student's performance is. Obviously this is most important for identifying those students who are in fact at the extremes, and less important information concerning those falling somewhere in the middle.
Helen Schinske