Little Evidence That "Brain Training" Has Wider Benefits
Digital “brain-training” games promising to improve users’ memory, focus or attention and stave off mental decline simply don’t have enough solid evidence to make their ambitious claims, a wide-ranging review of scientific literature alleges.
The new findings challenge a multi-billion dollar industry that only promises to grow as Baby Boomers age.
The review, published Monday in the journal Psychological Science in the Public Interest, finds “extensive evidence” that brain-training interventions improve performance on the tasks that users directly train for — remembering a string of numbers or letters, for instance — but less evidence that mastering these tasks can improve performance on related ones, such as remembering your grandchildren’s mobile phone numbers or their birthdays.
The researchers found little evidence that brain training enhances performance on distantly related tasks, or that training improves everyday cognitive performance.
Brain-training companies often cite research studies that say their product works, but the researchers found that many of the studies had “major shortcomings” in design or analysis. The researchers, led by Daniel Simons at the University of Illinois at Urbana-Champaign, looked at 374 scientific studies, including essentially every single one cited by a brain-training company. They found that none of the studies conformed to all of the best practices that researchers say are essential to drawing clear conclusions.
A few were simply too small, and others didn’t have suitable control groups. Others didn’t account for the so-called “placebo effect,” in which a person’s expectation for improvement might actually yield improvements, even if the training is ineffective.
“What the field really needs is studies that meet all of these best-practices standards,” Simons said in an interview. “None of the studies sufficiently meets these requirements.”
He stopped short of saying that brain training as a field is useless or that brain-training companies are selling what amounts to digital snake oil. “We just don’t know yet, because the studies have not been done in a way that allows you to draw clear conclusions.”
University of Virginia psychology professor Daniel Willingham, who was not involved in the review, said the field suffers from a placebo effect that’s “bigger than anybody would have guessed.”
Writing about the research on Monday, Willingham noted a recent study that nicely illustrates the problem: Researchers posted two kinds of fliers around a college campus, one offering a chance to participate in a study advertised as “Brain Training Cognitive Enhancement,” and another that advertised no stated purpose. Participants in both played brain games for an hour — too short for any possible benefit — yet those who thought they were training their brains showed improvement. The others, who completed identical training but did not expect to benefit, saw no improvement.
In spite of the field’s shortcomings, it is growing rapidly. A 2015 report said the total “digital-brain-health” market saw sales climb from $210 million in 2005 to $1.3 billion in 2013. It predicted that by 2020, cognitive/brain assessment and training software could see sales of nearly $3.4 billion.
Meanwhile, the industry has in a few cases run afoul of government regulators. Most recently, the U.S. Federal Trade Commission (FTC) last January said San Francisco-based Lumos Labs, creator of the popular Lumosity games, had engaged in “deceptive advertising” by suggesting their games could stave off memory loss, dementia and even Alzheimer’s disease. “Lumosity simply did not have the science to back up its ads,” the FTC said.
Lumosity agreed to settle a $50 million judgment by paying a $2 million fine — officials reduced the fine because of “financial hardship” on the company’s part — and Lumosity vowed to tweak its marketing materials. Its website now boasts 70 million brain trainers in 182 countries who “challenge their brains with Lumosity — and we’re honored to learn from this vibrant global community.”
In a statement issued Monday, Lumos Labs spokesperson Erica Perng said the company hasn’t quit the business by any means. “We strongly believe in the value of cognitive training, both as a consumer product and as a research tool,” she said.
Lumosity, Perng said, “draws on research that has developed over decades, but we are at the forefront of a new and rapidly innovating field, which means there is a lot we don’t yet know.”
She noted a large-scale, randomized, controlled study of 4,715 participants, led by a Lumos researcher in 2015, that found its brain training resulted in “transfer to a wide range of untrained measures of cognitive performance.” Perng said the study wasn’t included in the scope of Simon’s paper because it was published too late, but that he and his colleagues had acknowledged it as a “positive development.”
Simons conceded that the 2015 Lumos study was bigger than most, “and that’s a positive thing.” But he said it still suffers from a number of issues, including a “self-selection problem” — participants were expecting to get free brain-training sessions in the bargain. That actually keeps scientists from being able to answer whether brain training truly works.
“The goal here is to figure out what we need to do as a field to provide compelling evidence,” he said. “If there is going to be evidence.”
Follow Greg Toppo on Twitter: @gtoppo