The Seattle Times Investigative Story on Middle School Mental Health Screener

Will wonders never cease? For the second day in a row, the Times has printed a major story concerning public education in the Puget Sound region. And what was this story (all bold mine)?

The article is about the middle school mental health screener - Check Yourself - that districts around the region are using, including Seattle Schools. Written by reporter Daniel Gilbert, it's entitledHow King County’s efforts to help at-risk students created a record that could jeopardize their privacy.

I want to take this opportunity to do two things. 

One, a HUGE thank you to the parent who brought this to my attention. This parent is not from SPS and she did so much important research and was relentless in asking questions to many of those involved in the creation of the test. She wants to remain anonymous and I will honor that request.

Two, I read the entire article which was very in-depth and meaty and I want to say - I TOLD YOU SO. I wrote my first blog piece on this topic way back in 2018, trying to warn parents.  Here are the basics as Gilbert presents them:

The data collection is part of a roughly $30 million initiative funded by King County to screen middle- and high-school students for mental health, substance use and other risks. After screening more than 20,000 students over four years, county officials say the program is delivering on its promise, identifying previously unknown needs and saving student lives. 

But no one tells kids or their parents that they have helped fine-tune a commercial screening tool whose accuracy hasn’t been rigorously tested. No one tells them that contractors evaluating the program have a broader research agenda for the screening tool. And no one tells them that the information schools gather, without student names but with potentially identifying information, could become public, an investigation by The Seattle Times has found. 

The questionnaire collects students’ age, grade, race, language, gender identity, sexual orientation, school, and other details they include in open-ended responses — an unusually intimate record for a survey that isn’t anonymous. 

With such sensitive information in hand, schools must determine whether kids’ screening results are part of their education record. If so, parents would have a right under federal law to see things that their children might not want them to know. If not, the information isn’t protected by a federal privacy law — raising the prospect that it could be released publicly and compromise student privacy, experts say.

There are so many issues with this screener:

-I told the district it was likely they were in violation of PPRA ( Protection of Pupil Rights Amendment) in using this screener and SPS Legal said no. I got a couple of national public education advocates like Diane Ravitch to run some interference with the Department of Education and guess what? The DoE agreed with us and the district had to make some changes. 

What about FERPA? Well the article has a list of participating districts and some say it IS covered by FERPA and others say it ISN'T. I wonder who is right. 

The Snoqualmie parent emailed the school principal in March 2020 to ask for all the child’s SBIRT records for a “data privacy research project,” according to correspondence the district provided in response to a public-records request. 
 
School staff were not sure what to do. “We tell the students that this is not going to be shared with anyone, except for the counseling team,” one staffer emailed colleagues. 
 
Snoqualmie Valley told The Times that student responses to Check Yourself are “not identifiable by name,” and “staff do not maintain records of the student responses.” Yet three months after the Snoqualmie parent asked for the SBIRT records, the district provided a copy. 
 
“They know they have to,” said Rooker, a former head of the U.S. Department of Education’s Family Policy Compliance Office, who along with three other education experts believes student responses to Check Yourself are part of students’ education record.

- This part of the article is troubling because middle schoolers may not realize that they shouldn't put ANY identifying information on the screener:

Some students responded to open-ended questions with names of family members, friends and pets. (The Times did not attempt to identify any students.

Even without names, each piece of demographic information — such as the grade, ethnicity and school of a student who reports being captain of the soccer team — can dramatically shrink the universe of students to the point that they could be identified, according to privacy experts.  

“That feels to me like a significant gap that can result in real harm to students, especially considering the sensitivity of the questions that were asked and the responses that were elicited,” said Linnette Attai, a privacy consultant and author. 

Of greater concern, he said, is that some schools provide the anonymous screening data publicly, believing that removing names is enough to protect student identities.

“If I were a parent,” he said, “I would be very disturbed by that.”

- How long is data kept?

The data is purged from a central database every summer, according to the county, though it acknowledged that schools can export and save their own copies.

- What do experts say?

Interviews with 35 experts in screening questionnaires, research ethics and privacy yielded a complex mosaic of views on the program. Experts in screening youth generally lauded King County’s approach as innovative and thoughtful, while privacy professionals voiced alarm about the data collection. Several ethics scholars questioned whether the program crosses into research — despite a Seattle Children’s panel decision that it didn’t — which generally would require informed consent from students or their parents.

Others questioned why the county hired Seattle Children’s to evaluate the program when the institution has ownership rights to the screening tool at the heart of the program. That “should have raised some red flags,” said John Baumann, associate vice president for research compliance at Indiana University. “Why would they do that?”

- How we got here:

As King County laid the groundwork for its new screening program in 2017, Soukup consulted an expert on the subject, a professor at Seattle Children’s Research Institute named Cari McCarty. 
 
There are many free or low-cost questionnaires for screening kids, from mental health to substance use, that have been scientifically validated for accuracy. McCarty suggested such a questionnaire, but King County wanted one that covered a broader range than the off-the-shelf options. 

Flush with cash from the Best Starts for Kids levy — which voters last year expanded to $872 million over six years — and Mental Illness and Drug Dependency sales tax, the county had ample means to build its own program. 

King County fashioned its model on an approach known as SBIRT (Screening, Brief Intervention and Referral to Treatment, pronounced “ESS-birt”), which traditionally focuses on substance use. Soukup’s team settled on a questionnaire called “Check Yourself” that hasn’t been validated but is customizable.

McCarty knew Check Yourself well. She’d helped to create it.

Seattle Children’s owns the copyright to Check Yourself, according to a hospital lawyer’s email, and licensed it to Tickit Health, a Canadian firm that adapted the questionnaire for electronic use. McCarty herself stood to receive royalties from Tickit’s sales of Check Yourself, according to disclosures on academic papers.

In the Times' story, McCary claims the opposite - she's not received compensation in any form.  Hmmm.

- Was it research? The parent I worked with asked this question over and over and never really got a clear answer.

At the time, she was involved in at least four IRB-approved studies involving Check Yourself. The analysis for King County schools was different, according to McCarty’s plan, because her team was trying to improve the county’s program and not seeking to conduct research. The IRB was persuaded, but some experts disagree. 
 
“They’re trying to learn whether it works, whether it is able to identify at-risk middle school students and is therefore a useful and appropriate tool to use in middle schools,” said Michael Carome, director of Public Citizen’s health research group and a former senior official at the U.S. Office for Human Research Protections. “Those are factors that strongly point to this being research.” 
 
McCarty and her team submitted a paper last year to an academic journal about their work in King County. “I am very surprised,” one reviewer wrote, that the authors’ work was “determined to not be human subjects research and IRB approval was not needed.”

As schools began administering Check Yourself in 2018, some raised questions about the evidence behind it.
 
 
“We are unsure if we are being asked to field test a research or diagnostic tool,” staff at Tukwila School District wrote in an initial progress report to King County.

- Has use of the screener helped? The article gives some antidotal examples but my belief is that the privacy issues are much larger. 

Students, for their part, have reported that Check Yourself is easy to use. Tickit Health says its questionnaires are designed — through language, colors and icons — to build trust with various audiences and elicit more accurate information, and the approach appears to be effective with some students.

“I felt like it’s completely none of their business and totally irrelevant,” a Lake Washington School District parent, who asked that her name be withheld to protect the privacy of her kids, said of the gender and sexuality questions. She said that the screening quickly became the talk of the school, with students speculating about why their peers were pulled from class to meet with a counselor.

 I like the last paragraph in the article:

“Thanks for making us do this survey,” a Tahoma student wrote in response to an open-ended prompt. The student had initially felt uncertain, but after opening the questionnaire, “I felt like I could tell you guys anything that’s bothering me.”

Sounds hopeful, right? Except that this student used "making us" when, in fact, it's voluntary. And I'm sure the student thought he/she/them could say anything, likely believing it would stay private which might have encouraged them to give more information that should stay private. 

At the very end of the article, there are good questions and answers.  Here's a good one:

What changes might enhance protections for student privacy?

One interesting counterpoint to King County is Massachusetts, which passed a law in 2016 requiring schools to verbally screen students for substance use. The state uses a validated screening tool known as the CRAFFT-II. The same state law prohibits creating any record that includes information that could identify a student, a legal mandate that doesn’t exist in Washington. 
 
Other experts concerned about data privacy suggested that King County school districts should eliminate the school that students attend when transmitting data to Tickit’s central platform. That would effectively place anonymous students in a much larger population — the district rather than their school — and offer them greater privacy.

Comments

Unknown said…
You should do a little end zone dance, Melissa, and I credit you with bringing this to my attention years ago.

I didn't see terms like "SEL" or "social emotional learning" in the article either. Seems like a natural connection for ST to make.

These surveys have become instrumentalized to justify all kinds of programs, and SPS and my school and certain principal-targeting clubs at my school, love surveys, and they also love making claims on datasets they won't share. And when people ask for the dataset, the labeling of motives begins.

And then we don't consider how seriously the kids take them. I've seen sophomores have a blast with them, and the average time to complete the 50 question high school health behaviors survey is in the five minute range. That day is usually a social media period.

SP

Anonymous said…
Has use of the screener helped?

If a 13 year old reports that their mom's boyfriend is molesting them and schools are mandatory reporters, does the screener result in the boyfriend's prosecution?

If a middle school student reports that their single parent has an addiction that results in the student being neglected, what happens? The minor is referred for counseling? That doesn't seem sufficient.

Can law enforcement access Check Yourself survey responses? Say a 9th grader makes a threat against his high school, can the police request the 8th grade survey?

If a student confesses to a crime in an open-ended answer is the school obligated to turn that over to law enforcement?

If a student's parents are fighting for custody, can one parent request the student's education record, get a copy of the student's Check Yourself responses, then use the child's answers in court to argue that the other parent shouldn't get custody?

Can credit rating companies buy school-level responses and use that in determining that students who go to one middle school are less credit-worthy than students going to a different middle school based on the school-level results to the survey?

Can a school district decide that a particular student is undesirable and try to gently nudge them out the door, maybe slipping them a charter school brochure or maybe helping them find a better housing situation... in a different district.

You can see how one survey answered by one student would be of interest to lots of different people and entities in different ways. Parents have their interests. Counselors and healthcare workers have their interests. Schools have their interests. Private companies have their interests. Middle school is too young for children to be able to predict the consequences of their answering a questionnaire honestly. Especially when the school _says_ it won't be shared with anyone.

Simon Says
Simon Says, great questions.

I particularly would like to know what the district would do if a student was threatening to harm others.
Anonymous said…
The districts can't decide if the screening responses are covered by FERPA or not???!!! That's crazy.

Guinea Pigs
Anonymous said…
Along with every point raised by others, I am disturbed by the implication that SPS is prepared to ACT in a useful way on this data.I have little faith that SPS staff have the knowledge or bandwidth to respond well.

I'm also bothered by the school having access to information that parents do not have access to. Parents ought to have the right to know, or at least choose to let the information be confidential, if their child is at risk. As a parent, I did not sign up to have the school district act on the health needs of my child without informing me. The parent knows their history and seems like they should be the "first responder' for their kid. The whole exercise seems to assume that the parent would not act in the child's best interest. But then we are in Washington, where kids have full control of their health care at 13.

-Parents Care
Anonymous said…
NOTHING ABOUT THIS INVASIVE MENTAL HEALTH/LIFE STYLE ELECTRONIC QUESTIONNAIRE MAKES ANY SENSE, or, has any real-world beneficial value for the students taking it.

From an incredibly early age, I have taught my kids that EVERYTHING in the digital world is PUBLIC and PERMANENT. No ifs, ands, or buts.

I keep a 'social media fails' file to show my kids how innocent things can cost people their jobs, livelihoods, scholarships, university acceptances, friends, etc.

Anyway my kids declined to take the quiz. One got A LOT of pressure from their teacher. Really nasty.

IF THIS IS anonymous, then, let's say a kid's answers are troubling, well, how would they reach out to the kid to support or help?

Is the kid is suppose to self reflect after taking this? Realize they are in trouble or need someone to help and they are suppose to then reach out?

What? That makes no sense.

Who can see the data and 'use' the data? Where is the data stored? On who's servers? It matters. But then again, not really, because as long as it exists, really, anyone can get at it with some effort. How this evaded an Institutional Review Board for Human Subjects, I will never understand.

If this truly is anonymous, then, teachers won't know if a kid said they feel unsafe at home. That is damaging. A naïve 14 year old might think this quiz amounts to them bravely tip toeing toward seeking some help. When none comes, do they shrink back? Feel hopeless? Make even worse choices?

BESIDES, I would vehemently argue the teachers need to know their kids, by engaging with them and having a spidy sense about a kid that may be in trouble, not by some 30 min. bubble test. However realistically, I know, given today's reality in the classroom, that it is nearly impossible. In middle school, a teacher has kids for an hour a day, 150 of them. And during that hour, she's suppose to teach them math, so that they can learn it well enough to excel on some standardized test... And she is suppose to do that with a group of 30 kids, some of whom might have learning disabilities or an ELL background or come from economically deprived homes (things that can create barriers to learning despite great math brains), on top of this she is now suppose to be a social worker and spot kids who might be in trouble?

This quiz is pointless, and possibly destructive. It is not a way or a tool fit for the purposes of identifying a child at risk. But, not ironically, it may be a tool that will put a child a risk.

Teachers have an incredibly hard job. I respect their efforts.

This quiz, this researcher, reducing kids to a collection of dry data points, I have no respect for. She needs to get out into the real world of a classroom and see what it takes. Caring about kids is not something a quiz will aid.
BlueBus
Anonymous said…
The first thing I thought of when I first heard about this screener a couple of years ago was that it could be used by the district to mitigate its own liability. There is a huge conflict of interest for the district in the way this screener is handled.

We know from a 2004 study that about 1 in 10 students may receive "unwanted sexualized attention" during their K-12 years from public school employees, meaning anything from inappropriate comments to rape, and Seattle is no exception
(https://www.edweek.org/leadership/sexual-abuse-by-educators-is-scrutinized/2004/03)
(https://www.king5.com/article/news/local/seattle-ballard-high-school-sex-abuse/281-7692da13-c83b-4c71-add1-4c2335ea3bd1)
(https://www.q13fox.com/news/seattle-public-schools-employee-charged-with-molestation-of-elementary-student).

And then there are additional students who suffer non-sexual abuse that we hear of with disturbing regularity in the local press, such as school employees keeping children in cages (https://kuow.org/stories/locked-in-the-cage-report-finds-disturbing-discipline-measures-at-seattle-school),
pinching students, punching and shoving students (https://www.kuow.org/stories/seattle-schools-knew-these-teachers-abused-kids-and-let-them-keep-teaching),
inappropriate physical restraint (https://www.king5.com/article/news/hold-trauma/281-98706d74-8d2d-4bb0-a548-ddf76299d2de),
racist and unnecessary involvement of police (https://www.diversityinc.com/white-teacher-feels-threatened-filed-a-non-emergency-police-report-against-black-5th-grade-boy-in-seattle/),
racially motivated discipline (https://www.seattletimes.com/education-lab/race-dramatically-skews-discipline-even-in-elementary-school/)
and the like.

There are also many cases of abuse by omission, such as standing by or covering up while other employees abuse children (https://www.seattletimes.com/seattle-news/education/former-student-claims-seattle-school-district-silenced-him-from-speaking-up-about-hazing/),
not following IEP provisions (https://www.seattletimes.com/seattle-news/education/state-orders-fixes-at-seattle-schools-after-finding-special-education-violations-during-pandemic/) (https://www.seattletimes.com/seattle-news/education/u-s-department-of-education-launches-civil-rights-investigation-over-seattles-special-education-services-during-pandemic/),
failing to identify and support disabilities (https://www.seattletimes.com/seattle-news/education/seattlersquos-special-ed-mess-whorsquos-in-charge-of-what/)
and so forth.

School employees on the whole are obviously good, decent people who want nothing but the best for all kids, but that generalization doesn't mean the school district isn't using a screener like this to suss out its liability for its bad apples and potentially prevent perpetrators from meeting justice. The district should resolve its conflicts of interest with the screener or discontinue it.

Justice

Popular posts from this blog

Tuesday Open Thread

Why the Majority of the Board Needs to be Filled with New Faces

First Candidates for Seattle School Board Elections 2023