Saturday, July 30, 2016

Personalized Learning; Part Two

To start, I'm not against using computers to aid in teaching and learning.  But there are huge costs and unknowns and because of the investment of time and finances for this endeavor, parents need to ask hard, hard questions about what this will look like for their child.

I'm almost glad it took this long to get to Part Two because Data & Society put out a very good working paper about the topic: Personalized Learning: The Conversations We’re Not Having.  They start with this quote:
The Promise of Personalized Learning

“...if instead of having mass education as we now have, must have, with a curriculum, once we have outlets, computer outlets in every home, each of them hooked up to enormous libraries where anyone can ask any question and be given answers...then you ask, and you can find out, and you can follow it up, and you can do it in your own home, at your own speed, in your own direction, in your own time, then everyone will enjoy learning.

Nowadays, what people call learning is forced on you and everyone is forced to learn the same thing on the same day at the same speed in class. And everyone is different. For some it goes too fast, for some too slow, for some in the wrong direction. But give them a chance in addition to school — I don’t say we abolish school, but in addition to school — to follow up their own bent from the start...”

—Isaac Asimov, Interview with Bill Moyers, PBS, 1988
Is personalized learning really "new?"

Even without the use of new data-driven learning technologies, it is important to realize that every teaching environment is in some way personalized: everyday interpersonal interactions involve a degree of personalization as people respond to each other’s shifting moods by reading facial expressions (Walden & Ogan, 1988). In classrooms, teachers rely on these interpersonal cues, combined with their subject matter expertise, knowledge of how people learn, and knowledge of each student, to determine individual needs, adjusting their lessons in response to questions and behaviors (Brophy, 1985; Fredricks, Blumenfeld, & Paris, 2004; Wineburg, 2008).
 Their conclusion is actually a good beginning:
This primer highlights the tensions between what is being promised for personalized learning and the practical realities. The realities do not point to a binary conclusion of whether personalized learning is beneficial or not, but rather a complex story in which technology developers are applying successful marketing tactics (personalized content delivery and recommendation systems) to education, and start-up models to the learning experience (risk-taking, test in field), while administrators seek to improve efficiency and performance through formative testing and tailored learning modules. 
Although they also have a strong stake in the debate, the perspective of parents and teachers are not often included in these discussions. Potentially, personalized learning systems can empower teachers by providing scalable data on student performance, interests, and behaviors, yet they can also disempower through opaque processes and prescriptive formats. For parents, the systems potentially offer increased communication about their child’s schooling. A key issue is that personalized learning currently isn’t being used as well as it could be and the reason is that few, if any, of these actors hold a complete picture of how to maximize use of student data to benefit learning. Should parents focus on incremental scoring or modular completion? Should their efforts to support their children focus on raising those numbers? Which data points are useful for teachers in real time, and how would they know? Do technology developers have a responsibility for testing learning programs before marketing to schools?
But let's review.

Personalized learning does not have any specific definition but basically is a more one-on-one learning that the student does alone as developed by either the teacher, adaptive software or both.  Meaning, it is non-linear learning with students going at their own pace.  Many districts/schools are using this about an hour a day, mostly on a computer.  (I haven't yet found a study about developmental appropriateness before 3rd grade.  The Rocketship charter group is big on this use and part of that may be because you don't need a certified teacher in the room (just a "facilitator") which cut costs.

From Data&Society:

Some types of student data are restricted by state and national laws and/or industry good practice standards, but the approach being used for data-driven or adaptive personalized learning is the same as the one that recommends purchases on Amazon or movies on Netflix. For many personalized learning systems, student data such as age, gender, grade level, and test performance are analyzed against idealized models of student performance, or students of the same background or class, or nationwide pools of grade and/or competency level. A profile is created for each student that typically categorizes her or him as part of a group that performs similarly or demonstrates shared interests or demographics. Then, data-driven content recommendations are sent either directly to the student or to the teacher for further intervention.

Page 3 of this paper has an excellent wheel of "personalized learning terms used in marketing materials and media" that you might take a look at as I believe you will be hearing them more and more.

 Much of any of the success of personalized learning will have to come from teachers and principals and that depends on resources and training.  They will have to become experts in data analysis in order to be able to read assessment results and understand how each student is doing.

The Center on Digital Education did a study and reported this:
But while personalized learning is certainly promising, a recent CDE survey of 215 IT leaders in K-20 education shows the concept has not been widely implemented in K-12 or higher education. Just 20 percent of K-12 respondents and 15 percent of higher education respondents reporting having created a personalized learning culture.
Of course, there is money to be made from personalized learning and all the structure and planning it takes.  Mark Zuckerberg of Facebook seems to be all in.  From the Center on Digital Education:
Developing new software for K-12 schools. Investing in hot ed tech startups. Donating tens of millions of dollars to schools experimenting with fresh approaches to customizing the classroom experience. 

All are part of a new, multi-pronged effort by Facebook founder and CEO Mark Zuckerberg and his wife, pediatrician Priscilla Chan, to use their massive fortune to reshape public education with technology.
We don't know for certain that it's going to work," he said. "All we can really hope to do is provide an initial boost and try to show that this could work as a model, and hopefully it gets its own tailwind that carries it towards mainstream adoption." 
In 2014, Facebook assigned a team of its employees to work with a California charter school network known as Summit public schools. Together, the engineers and educators are developing a digital platform called the PLP, short for Personalized Learning Plan.
But Facebook and Summit have also made clear they hope to eventually make the PLP available to every K-12 school in the country, and their contract leaves the door open for Facebook to commercialize the tool in the future.
" use their massive fortune to reshape public education with technology."  Just like Gates before him, Mr. Zuckerberg was not hired, appointed or elected to this work and it is highly unlikely his own children will ever set foot in a public school.

Because of the enormous investment and far-reaching work this will all take, plus the push from wealthy philanthropists, I believe districts should take a "pilot" approach to this idea.  

Naturally, there are other issues.  This from the Center on Digital Education about what Florida is doing:
"It's not about putting students on computers. It's not about having students learning by themselves, where they have a packet of information," said Kathy Halbig, the school district's coordinator of personal learning for students. "The whole idea of personalized learning is meeting students at their level.
But the national push for more "competency-based education," as the effort is also known, has prompted harsh criticism from some parents and school advocates who fear it will lead to more testing and more online instruction, to the benefit of corporations selling software and the detriment of students.

"The testing industrial complex is not giving up easily, working eagerly to establish what could mean testing every day. It is called competency-based education," wrote Stephen Krashen, a professor emeritus at the University of Southern California and education activist, on his blog last month.

He went on to call the effort a "a radical and expensive innovation" to replace "regular instruction" with online learning.
Plus there is always one issue that is dear to my heart - student data privacy.  From Center for Digital Education:
School districts need rich data sets in their technology systems to power personalized learning. But this type of data collection will continue to clash with the public's concern over privacy, Pane said.
Though more than 25 student data privacy bills were signed into law in 2014 at the state level, administrators will need to wrestle with data privacy and protection as they sign contracts for technology systems. And that means building trust with parents and applying state and federal laws to their decisions.
From the Data&Society paper:
Business models and technology development approaches to collect as much data as possible to determine what is useful clash with parents’ and advocacy groups’ desire to know what types of data are necessary. Much uncertainty and opacity surround the present and future use of these datasets. What data are necessary to develop effective personalized learning systems remains an open question. Data collection is not limited to proficiency assessments or demographic information provided by the schools, but is increasingly extending to students’ personal lives via school-issued devices brought home, and the monitoring of students’ social media accounts (Singer, 2015; Quinton, 2015). Researchers question when such student data collection shifts from instructional benefit to oversight and surveillance (Selwyn, 2014; Reddy, et al., 2015).
Well, FERPA was written in 1974 with no significant updates - save Arne Duncan opening the door for non-educational entities to have access to student data in 2011.  Raise your hand if you think things have changed for students and classrooms since 1974.   

A good, if short, article about a study done by the Brookings Institute.  The results were good; gains on testing and movement in the achievement gap.  However:
Nearly every school employs some aspect of personalized learning as they define in the RAND study. For example, a component of personalized learning is an emphasis on college and career readiness and incorporating data from formative assessments. For many schools around the country these are common practices. Only certain aspects of personalized learning, for certain schools, represents a departure from business as usual. To some extent personalized learning is nothing new.
A more concerning issue is the generalizability of the findings. Each school in the treatment group received funds from the Next Generation Learning Challenges, the Charter School Growth Fund, or the Bill and Melinda Gates Foundation. Schools gained financial support for personalized learning through a competitive grant process. 57 out of the 62 (91 percent) schools in the treatment sample were charter schools compared with four percent of schools in the United States. This study allows for an inference about the influence of personalized learning on student achievement in charter schools, but on the assumption that students across these school types are qualitatively different in some way, these findings cannot support a claim about traditional public school implementations.
More from teh working paper from Data&Society that I mentioned at the top of the thread: 

Page 5 - a good discussion about Common Core standards "as a counterpoint to competency-based education." 

Page 6
Adaptive systems aim to mirror and support the learning process, which isn’t fixed and does not necessarily have a single end goal. Responsive systems are more limited, essentially offering an interface to pre-determined content, like a hyper-linked menu or a series of digital buttons. In comparison to truly adaptive systems, responsive systems are further from the neurological processes of teaching and learning, offering something much closer to an interactive textbook than a tutor.
Much of the promise hyped by investors and enthusiasts revolve around the image of the truly adaptive private tutor, even if what is more commonly delivered is simply responsive. SmartSparrow Founder and CEO Dror Ben-Naim observes that "Many of the so- called ‘adaptive learning' platforms are really more like content recommendation systems -- like Amazon or Netflix. I don't see where the learning is adaptive. The content is not changing in response to the students" (Waters, 2014).
Page 7 - talks about Class Dojo, Schoology and other platforms.
Rather than tailoring particular content based on student competencies, these systems serve more of a tracking and organizing role.
Page 11 - the real ugly of personalized learning
While the responsiveness of personalized learning systems hold promise for timely feedback, scaffolding, and deliberate practice, the quality of many systems are low. Most product websites describe the input of teachers or learning scientists into development as minimal and after the fact (Guernsey & Levine, 2015). Products are not field tested before adoption in schools and offer limited to no research on the efficacy of personalized learning systems beyond testimonials and anecdotes.
Page 12 - what about the "personal" in learning?
Praise for personalized learning systems often focuses on student performance and how promised improvements will benefit school systems and student grade progression. Less attention is paid to how this focus on performance and the individualization of the learning experience may impact a student’s psychological well-being. Self-determination theory posits that students have basic psychological needs that must be met for optimal well-being, which include feelings of competence (confidence that they can achieve academic goals), autonomy (belief that they have choice and independence in identifying and pursuing goals), and relatedness (development and preservation of close personal relationships) (Deci & Ryan, 2000; Currie, et al., 2012).3 
There is a value in collaborative work, in learning teamwork skills, and in learning to communicate, that is not readily apparent in current personalized learning systems. None of the demos or descriptions of personalized learning reviewed for this primer mentioned verbal interaction, or address students’ need for relatedness.
Page 15 - Does it really work?

Despite decades of standardized assessment, researchers find no clear feedback loop between assessment data and informing instructional practice (Militello, et al., 2013; Selwyn, 2015). While used for ranking and accountability purposes, the potential for these large-scale collections for improving student learning outcomes remains unclear (Militello, et al., 2013). Since many personalized learning systems are based on the large-scale gathering and analysis of data, it becomes important to determine how effective such practices are at supporting instruction and improving student learning outcomes.  
The benefits of learning analytics are more apparent in higher education, where larger scale tests are possible within a single course (500 students per large lecture class instead of the usual 25-30 in an elementary classroom). Benefits are most frequently seen in well-defined subjects (with clear right or wrong answers) such as math and science (Nye, Graesser & Hu, 2014). The benefits of data in informing instruction in K-12 are less clear.
In 2011, the Department of Education found that a majority of K-12 teachers had received little to no training in data interpretation (Department of Education, 2011). Yet a more recent study suggests improvement. In 2015, a Gates study of 4600 K-12 teachers showed that a nearly even split between those who had difficulty interpreting data reports and those who found them useful (Bill & Melinda Gates Foundation, 2015).
Page 18:
Underlying adaptive personalized learning systems are algorithms—analyses driving programs to serve content that increases the likelihood of reaching a desired end goal. But which goals are being encoded in the design of personalized learning systems? Multiple goals are described in marketing materials (e.g., improved scores on quizzes or preparation for Common Core), yet optimizing for multiple goals is ineffective. It is currently unclear from descriptions of personalized learning systems, what goals each are optimizing for, and how they are differentiating between interim goals (e.g., testing to represent mastery) and larger end goals (progressing to the next grade level).
I did find a very good analysis about the work to be done for districts who want to have a bigger personalized learning footprint from Fulton County Schools which is a highly-regarded district in East Point, Georgia.  Apparently their district started on this road in 2013.    This is an informative PowerPoint and I really liked page 10, where they ask good questions about ALL the issues:

- Curriculum - What will your students learn?
- Learning - How will your students learn?
- Tools and Support: What Resources do you need?
- Operation: How will you run it?

I wish SPS used this matrix on all their projects.

 Page 12 shows " Current State analysis of FCS district capabilities," a kind of "gut check" of where their district is.  Again, I appreciate the honesty in this page about how the work is truly aligning with resources and teaching/learning.  But they admit that only 21% of their principals believe that there has been enough PD for this effort.  And they say:
FCS has proven success in planning and implementing pilots and rollouts (e.g., rollout of Charter System Framework, Standards-based Grading pilots, Amplify pilot, etc). However, as cited by participants in other pilots, FCS has not been successful at clarifying what success looks like and collecting the relevant data needed to determine if success has been achieved.
That takes some humility to make that admission.  But the on-going slides explain the enormity of the project.  This is not a one-off project.  It is a fundamental shift in teaching and learning.  What's interesting on page 44 - which is list of areas that personalized learning could support - is no mention of "equity" which seems to be the new by-word for work in SPS.


stuart jenner said...

A first question is "how much screen time is appropriate for each child per day, per week, or at one sitting without a break for their eyes." Adults who work a lot with computers in the workplace have a tendency to develop repetitive strain injury, carpal tunnel, back and musculoskeletal problems, so that's also an issue we should be looking at.

A second question is how brain development works. In essence, our brains our wired in certain ways. Computers and screens result in very different interactions than has been the case with traditional brain development. As an example, there have some news stories recently about the importance of taking notes by hand, not on a computer.

A third question is whether schools that are under a lot of pressure to show return on investment in technology have the flexibility and freedom to say "this is the limit of how much computer use is appropriate." There were some stories about how high-scoring PISA countries did not have a lot of computer use.

Unfortunately, much of the time the premise seems to be: tech good, complainers bad, just get out of the way, your kids have ample access and you just want to increase the digital divide. So go away. But in working with elementary math club students, which is a very small sample set, I saw kids who had "mastered" a skill such as adding mixed fractions using ST Math, but could not solve the problems mentally or with paper and pencil.

Melissa Westbrook said...

Good points, Stuart.

Anonymous said...

We have had success using Dreambox in math. It is just a small part of the math block at school and each child is at their own stage. I know it didn't work for some kids in the class because they would get frustrated/bored/distracted by other features. But the advantage of being able to use it at home allows parents to see where they kids are and can help make learning fun.

NE mom

Stuart Jenner said...

Key points from the post above: fun, one of many tools, parents can see what's going on and discuss, and see if something is working for their child.

Earlier this year, I did a lot of research trying to find any type of research about just how much screen time was appropriate. I could not find definite answers. The UW and Seattle Children's have various people working on this subject, some of whom are quoted in the movie "Screenagers."

But truly unbiased research doesn't yet seem completed. So in a sense, our kids are in one big uncontrolled experiment.

There's definitely a lot of money in educational software and hardware (ipads, Chromebooks, Windows). I find it hard to believe any of the philanthropists from Silicon Valley or Redmond who care so much about educational reform are going to fund studies that might lead to the conclusion that kids are better off with more traditional methods, but maybe in a tracked classroom setting as opposed to differentiation within a classroom.

I am hoping some of the experts from the UW and Children's can chime in with latest findings, or studies in progress. I don't want to be too cynical here. Critical evaluation is possible. A person from Seattle Children's did some research on tv shows for kids that Disney owned, and basically put them out of business. Remember Baby Einsteins?,8599,1650352,00.html

Anonymous said...

I find, personally, that the inherent distractibility of working online versus in a classroom or non-computer setting is nothing less than devastating. Though it would probably be a good skill for kids to learn! Staying focused on the problem at hand and not taking a quick look at blogs etc ;-) would be a good lesson in delayed gratification.

I already see my teen struggling with this when doing homework online. It is too easy to click over to your favorite you-tuber....

-SPS parent

Sanderson said...

"In 2015, a Gates study of 4600 K-12 teachers showed that a nearly even split between those who had difficulty interpreting data reports and those who found them useful (Bill & Melinda Gates Foundation, 2015)."

So, if you didn't find the data provided useful, it could only be because you don't know how to interpret the data? I'd have to look at the research itself, but that is what the quote above seems to be insinuating. I find this assertion unlikely. I would venture to say that there are many educators that are perfectly capable at looking at data, but still find the results useless in regard to the realities of an actual classroom environment.

Ms206 said...

I completely agree with stuartjenner that computerized learning raises its own set of issues.

I teach children with autism in elementary school and I find that the iPads and desktop computers are great for kids to use as a center during reading instruction or math instruction. The computers can also be a good incentive or choice activity that I can offer students for positive behavior. The kids can access programs like Lexia or First in Math, which provide individualized practice. When I had to do the state alternate assessment for my third grade student (because they don't take the regular standardized tests), I had my other students working mainly on the computers so that I could do the testing efficiently. Thanks to the computers, and a class size of 6 students, I was even able to test while I was the only adult in the room (when my assistant took his lunch), although there were still interruptions during testing when I had to help students if they did things like press the wrong button on the computer or if they needed help tying in a web address.

Sometimes my students would ask me, "Do we get to use the computers today?" And on some days, no, they didn't use the computers during instructional time because we had other activities. I fear that this younger generation will become "computer dependent" and won't know how to do anything without computers. So it's important to teach computer and technology skills, but kids also need to learn how to do things the old-fashioned way.

Greg said...

Great writeup, Melissa. I couldn't agree more that personalized learning should have a high bar. It should be measurably much better before widely adopted.

The goal is to solve the 2 sigma problem without having to use 1:1 human tutoring, so there should be obvious, objectively measurable increases in test scores if it's anywhere close to achieving its goal.

Unfortunately, I think you're right that, as you mention a few times, people are trying to peddle things that don't actually work as "personalized learning". Keeping the goal in mind may help here. If it doesn't make substantial and obvious progress on the 2 sigma problem, it shouldn't be adopted.