Don’t believe online reviews of health products

February 10, 2017

This is the first ever study to compare reviews to controlled clinical trial data
The research confirms that reviews rarely conform to medical science

Mia De Graaf For Dailymail.com

Published:
19:00 EST, 9 February 2017

|
Updated:
19:23 EST, 9 February 2017

e-mail

View
comments

Scientists are urging people to stop buying health products based on online reviews.

For the first time ever, a study has compared clinical trial data with user-generated online reviews.

The results, perhaps unsurprisingly, reveal a staggering difference between opinion and science.

And yet, reviews are one of the most important factors that drive product sales in today’s era of e-commerce.

Reviewers on weight loss pill Orlistat lost about twice as much weight (30.8lbs) as those in clinical trials (15.4lbs), according to a new study by the University of Aberdeen

‘These narratives have a powerful influence on our own future health behavior because they provide simple and clear anecdotes, but this study shows that they can be very misleading,’ warns Dr Mícheál de Barra, of the University of Aberdeen.

Dr de Barra examined more than sixteen hundred online reviews of weight loss pills and high cholesterol treatments on Amazon.com.

He found the reviews portrayed the products in a far more positive light than the clinical trial data would suggest.

For example, the majority of Benecol users on Amazon claimed to experience a drop in cholesterol after using the product.

However, less than a third of users found the same in carefully controlled trials.

Similarly, reviewers on weight loss pill Orlistat lost about twice as much weight (30.8lbs) as those in clinical trials (15.4lbs).

Dr de Barra insisted his study does not discredit these products.

Not got a university degree? Then you’re MORE likely to…

Antibiotics taken during pregnancy weaken a baby¿s immune…

Merely, it shows the need for caution when selecting products.

‘These treatments are not entirely ineffective. However, what we show is that the reputation as described in these reviews is much more positive than the clinical trial data show.’

People need to be aware, he warns, that the only people who leave reviews are ones who are driven to do so by an extremely good experience or an extremely bad one.

‘Only some people who try a treatment will then go on to tell other people about their experience, however, this subset of people are usually only those who have good outcomes.

‘So, you hear a friend of yours had a good result using a treatment of some kind, and you think “well maybe this works”.

‘Your friend is probably not lying – but the problem is that people with average or poor outcomes don’t tend to share their experiences.

‘This means you get a positively skewed view of the treatment.’

Dr de Barra adds that his study also opens up the often ignored discussion about medical overuse.

Medical overuse is estimated to cost the $226bn in the US alone, and patient demand for medicines with limited value is one important cause.

Dr de Barra warns this study shows how a demand for ineffective medicines can easily develop when people rely on hearsay alone.

‘Realistically,’ he says, ‘it is ridiculous to think that every health decision we make will be informed by systematic reviews – people have lives to lead!

‘This study shows though that it’s important to be aware of the biases that can make informally acquired information unreliable.’