The price of ignorance: the Durham study and research ethics

Ben Goldacre (who seems to be one of this blog's favorite sources) tears into the Durham fish oil trial. A while ago Durham County together with the company Equazen decided to test whether giving omega-3 supplements would improve the GCSE scores of children. Unfortunately there were clear problems with the trial design. In the face of criticism the organisations involved refused to give out information on the experimental setup and even claimed not to be running it as a trial (despite numerous statements to the press). GCSE scores did not generally increase. Despite this, now positive results are claimed - largely because what is measured has been changed to suit the data. The most vexing thing about the whole affair is that the trial could have been done in a proper manner for the same amount of money.

Medical studies are rigorously controlled in order to safeguard the rights and well-being of the subjects. Studies on children are particularly sensitive. In the Durham case one could probably complain that it was not up to the standards of most such studies - did the pupils and parents really give informed consent? Would they have approved of the study if they knew how it would be presented in the media? Fortunately the level of risk seems to have been minimal and clearly the children had a choice in taking the supplements or not. The people responsible also likely thought they were doing something helpful. That does not absolve them from blame, but the problem is not an unethical study in itself.

The reason so much effort normally goes into designing scientific trials is to maximize the amount of information that can be gained about while reducing the risk of being misled by the myriad of other factors that affect the investigation. Part of this is just cost-effectiveness, but there is also an ethical underpinning: spreading misleading or mistaken information impairs science and decision-making. Fraud and bias hurt other research by introducing errors that can only be corrected by doing even more research showing it to be in error. But it is very hard to avoid bias. Among many other bias sources, the desire to see one's pet theory vindicated often fool even conscientious researchers into subtle mistakes. Results also tend to be biased towards the interests of the funding source.

In the case of the Durham trial it is clear from their statements that the people involved have a bias towards the theory that supplements are good and (in the case of the company) have a financial interest in promoting it. The study had serious methodological flaws that would have been a problem in any case, but the data was interpreted in a way that seems to bias it towards the desired conclusion. Not only have the results been presented in the media as a success, but at least one (premature!) reference to the study has crept into a scientific paper in a renowned journal (Fernando Gómez-Pinilla, Brain foods: the effects of nutrients on brain function, Nature Reviews Neuroscience 9, 568-578 July 2008) where it may mislead readers.

One can also criticise the study from a marketing ethics perspective. Ethical marketing involves taking responsibility for products and decisions, dealing fairly with stakeholders, respecting consumer rights and safeguarding the interests of vulnerable consumers, such as children. The flip-flopping over whether it was a trial or not shows a problem with taking responsibility. The refusal to give information about the study goes against dealing fairly with stakeholders, who in the case of public health and education clearly involve not just researchers but the public at large. There is also a possible issue of using the children of Durham as advertising; while nobody's privacy seems to have been infringed some might have reservations against being promoted as arguments for a particular product - especially when the argument is weak.

To me the greatest annoyance with the whole affair is that it missed the chance of doing good research on an important question. There are reasons to think certain fatty acids can have a positive effect on developing brains. If they have, the social gains of ensuring adequate uptake could be very large. Yet we have very little solid data. This is something the Durham study could have helped answer. Now the most likely result will be bringing the area into disrepute.

One might question whether the close cooperation between a public institution and a commercial company was appropriate. But the study would not have been possible without the cooperation, and addresses a question of public interest. If the study had simply been handled better, perhaps with outside help to safeguard against bias and to make a proper study, the result could well have been both to the benefit of the company, the county and the public. Now the opposite was achieved.

Comments

[Dr* T]

"If the study had simply been handled better, perhaps with outside help to safeguard against bias and to make a proper study, the result could well have been both to the benefit of the company, the county and the public"

There were offers (even from Ben himself, I believe) to help with this trial. I'm afraid I'm more sceptical with regards to the aims and potential achievements - no nutrapill company is going to gamble £1 million of product if there is any doubt in the result and the raft of other publications would have lead them to believe that PR rahter than scientific rigour was the way to go.

The chap in charge (Dave Ford) in my opinion has allowed his position on the council to be abused by a private company, has allowed mass pill-peddling on kids (with no possibility of finding out anything new) and has promoted the idea that pills solve complex social problems.

I'm amazed he still has his job.

[Kess]

If a pharmacutical company wanted to try out some experimental mind-altering drugs (for how else can one describe the claimed effects of concentrated fish oil capsules on brain function) on children there would be outrage.

Yet it's OK if the pills are "natural" and produced by some nice friendly people from a little cottage industry (worth £billions) that is, of course, only concerned about the health and welfare of our children.

Strange but true.

[Anders Sandberg]

People are generally far more accepting when something is branded as natural or produced by an "underdog" than they should rationally be. For example, most people are queasy about using a therapy to change gene expression in order to enhance the memory of children, but are quite ready to have mothers supplement with choline or choline-rich food when told it can likely improve children's cognition - but the two things are exactly the same thing.