The Seattle Times Investigative Story on Middle School Mental Health Screener

Will wonders never cease? For the second day in a row, the Times has printed a major story concerning public education in the Puget Sound region. And what was this story (all bold mine)?

The article is about the middle school mental health screener - Check Yourself - that districts around the region are using, including Seattle Schools. Written by reporter Daniel Gilbert, it's entitled, How King County’s efforts to help at-risk students created a record that could jeopardize their privacy.

I want to take this opportunity to do two things. 

One, a HUGE thank you to the parent who brought this to my attention. This parent is not from SPS and she did so much important research and was relentless in asking questions to many of those involved in the creation of the test. She wants to remain anonymous and I will honor that request.

Two, I read the entire article which was very in-depth and meaty and I want to say - I TOLD YOU SO. I wrote my first blog piece on this topic way back in 2018, trying to warn parents.  Here are the basics as Gilbert presents them:

The data collection is part of a roughly $30 million initiative funded by King County to screen middle- and high-school students for mental health, substance use and other risks. After screening more than 20,000 students over four years, county officials say the program is delivering on its promise, identifying previously unknown needs and saving student lives. 

But no one tells kids or their parents that they have helped fine-tune a commercial screening tool whose accuracy hasn’t been rigorously tested. No one tells them that contractors evaluating the program have a broader research agenda for the screening tool. And no one tells them that the information schools gather, without student names but with potentially identifying information, could become public, an investigation by The Seattle Times has found. 

The questionnaire collects students’ age, grade, race, language, gender identity, sexual orientation, school, and other details they include in open-ended responses — an unusually intimate record for a survey that isn’t anonymous. 

With such sensitive information in hand, schools must determine whether kids’ screening results are part of their education record. If so, parents would have a right under federal law to see things that their children might not want them to know. If not, the information isn’t protected by a federal privacy law — raising the prospect that it could be released publicly and compromise student privacy, experts say.

There are so many issues with this screener:

-I told the district it was likely they were in violation of PPRA ( Protection of Pupil Rights Amendment) in using this screener and SPS Legal said no. I got a couple of national public education advocates like Diane Ravitch to run some interference with the Department of Education and guess what? The DoE agreed with us and the district had to make some changes. 

What about FERPA? Well the article has a list of participating districts and some say it IS covered by FERPA and others say it ISN'T. I wonder who is right. 

The Snoqualmie parent emailed the school principal in March 2020 to ask for all the child’s SBIRT records for a “data privacy research project,” according to correspondence the district provided in response to a public-records request. 

 

School staff were not sure what to do. “We tell the students that this is not going to be shared with anyone, except for the counseling team,” one staffer emailed colleagues. 

 

Snoqualmie Valley told The Times that student responses to Check Yourself are “not identifiable by name,” and “staff do not maintain records of the student responses.” Yet three months after the Snoqualmie parent asked for the SBIRT records, the district provided a copy. 

 

“They know they have to,” said Rooker, a former head of the U.S. Department of Education’s Family Policy Compliance Office, who along with three other education experts believes student responses to Check Yourself are part of students’ education record.

- This part of the article is troubling because middle schoolers may not realize that they shouldn't put ANY identifying information on the screener:

Some students responded to open-ended questions with names of family members, friends and pets. (The Times did not attempt to identify any students.) 

Even without names, each piece of demographic information — such as the grade, ethnicity and school of a student who reports being captain of the soccer team — can dramatically shrink the universe of students to the point that they could be identified, according to privacy experts.  

“That feels to me like a significant gap that can result in real harm to students, especially considering the sensitivity of the questions that were asked and the responses that were elicited,” said Linnette Attai, a privacy consultant and author. 

Of greater concern, he said, is that some schools provide the anonymous screening data publicly, believing that removing names is enough to protect student identities.

“If I were a parent,” he said, “I would be very disturbed by that.”

- How long is data kept?

The data is purged from a central database every summer, according to the county, though it acknowledged that schools can export and save their own copies.

- What do experts say?

Interviews with 35 experts in screening questionnaires, research ethics and privacy yielded a complex mosaic of views on the program. Experts in screening youth generally lauded King County’s approach as innovative and thoughtful, while privacy professionals voiced alarm about the data collection. Several ethics scholars questioned whether the program crosses into research — despite a Seattle Children’s panel decision that it didn’t — which generally would require informed consent from students or their parents.

Others questioned why the county hired Seattle Children’s to evaluate the program when the institution has ownership rights to the screening tool at the heart of the program. That “should have raised some red flags,” said John Baumann, associate vice president for research compliance at Indiana University. “Why would they do that?”

- How we got here:

As King County laid the groundwork for its new screening program in 2017, Soukup consulted an expert on the subject, a professor at Seattle Children’s Research Institute named Cari McCarty. 

 

There are many free or low-cost questionnaires for screening kids, from mental health to substance use, that have been scientifically validated for accuracy. McCarty suggested such a questionnaire, but King County wanted one that covered a broader range than the off-the-shelf options. 

Flush with cash from the Best Starts for Kids levy — which voters last year expanded to $872 million over six years — and Mental Illness and Drug Dependency sales tax, the county had ample means to build its own program. 

King County fashioned its model on an approach known as SBIRT (Screening, Brief Intervention and Referral to Treatment, pronounced “ESS-birt”), which traditionally focuses on substance use. Soukup’s team settled on a questionnaire called “Check Yourself” that hasn’t been validated but is customizable.

McCarty knew Check Yourself well. She’d helped to create it.

Seattle Children’s owns the copyright to Check Yourself, according to a hospital lawyer’s email, and licensed it to Tickit Health, a Canadian firm that adapted the questionnaire for electronic use. McCarty herself stood to receive royalties from Tickit’s sales of Check Yourself, according to disclosures on academic papers.

In the Times' story, McCary claims the opposite - she's not received compensation in any form.  Hmmm.

- Was it research? The parent I worked with asked this question over and over and never really got a clear answer.

At the time, she was involved in at least four IRB-approved studies involving Check Yourself. The analysis for King County schools was different, according to McCarty’s plan, because her team was trying to improve the county’s program and not seeking to conduct research. The IRB was persuaded, but some experts disagree. 

 

“They’re trying to learn whether it works, whether it is able to identify at-risk middle school students and is therefore a useful and appropriate tool to use in middle schools,” said Michael Carome, director of Public Citizen’s health research group and a former senior official at the U.S. Office for Human Research Protections. “Those are factors that strongly point to this being research.” 

 

McCarty and her team submitted a paper last year to an academic journal about their work in King County. “I am very surprised,” one reviewer wrote, that the authors’ work was “determined to not be human subjects research and IRB approval was not needed.”

As schools began administering Check Yourself in 2018, some raised questions about the evidence behind it. 

 

“We are unsure if we are being asked to field test a research or diagnostic tool,” staff at Tukwila School District wrote in an initial progress report to King County.

- Has use of the screener helped? The article gives some antidotal examples but my belief is that the privacy issues are much larger. 

Students, for their part, have reported that Check Yourself is easy to use. Tickit Health says its questionnaires are designed — through language, colors and icons — to build trust with various audiences and elicit more accurate information, and the approach appears to be effective with some students.

“I felt like it’s completely none of their business and totally irrelevant,” a Lake Washington School District parent, who asked that her name be withheld to protect the privacy of her kids, said of the gender and sexuality questions. She said that the screening quickly became the talk of the school, with students speculating about why their peers were pulled from class to meet with a counselor.

 I like the last paragraph in the article:

“Thanks for making us do this survey,” a Tahoma student wrote in response to an open-ended prompt. The student had initially felt uncertain, but after opening the questionnaire, “I felt like I could tell you guys anything that’s bothering me.”

Sounds hopeful, right? Except that this student used "making us" when, in fact, it's voluntary. And I'm sure the student thought he/she/them could say anything, likely believing it would stay private which might have encouraged them to give more information that should stay private. 

At the very end of the article, there are good questions and answers.  Here's a good one:

What changes might enhance protections for student privacy?

One interesting counterpoint to King County is Massachusetts, which passed a law in 2016 requiring schools to verbally screen students for substance use. The state uses a validated screening tool known as the CRAFFT-II. The same state law prohibits creating any record that includes information that could identify a student, a legal mandate that doesn’t exist in Washington. 

 

Other experts concerned about data privacy suggested that King County school districts should eliminate the school that students attend when transmitting data to Tickit’s central platform. That would effectively place anonymous students in a much larger population — the district rather than their school — and offer them greater privacy.