This week a widely-criticised paper claiming negative effects of Roundup and Roundup-tolerant genetically modified maize on rats was retracted by the journal Food and Chemical Toxicology. The author, Professor Seralini, stands by the results, but received a letter from the journal stating that “The panel had many concerns about the quality of the data, and ultimately recommended that the article should be withdrawn.”
Criticisms included inadequate numbers of rats to draw conclusions (especially as the strain used were extremely prone to tumours). There were also insufficient control groups (there should have been more rats fed on non-GM diets to see whether this was any different to those fed GM diets). The stats have also been questioned.
These criticisms were widespread from the scientific community, many from universities, including toxicologists who have no interest (vested or otherwise) in GM. I have included some responses to the study from scientists and food regulatory authorities at the bottom of the post.
There were also animal welfare issues. Even to my untrained eye, it is clear from the pictures that the rats should never have been kept alive with tumours which had become so large. The French Society of Toxicologic Pathology (SFPT), a not for profit organisation specialising in veterinary and toxicologic pathology, wrote to the editor with serious animal welfare concerns.
Seralini, to his credit, tackled many of the criticisms he faced, and this was published in the same journal. I do not know enough to respond to most of the toxicologocal points, but I must say his animal welfare points concerned me.
There is one thing Seralini and I agree on. He said: “[the study] should not be considered as a final point in knowing the toxicological effects of NK603 and R[oundup].”
I hope this proves to be true, that this study is repeated by independent scientists, using an improved experimental design and with higher standards of animal welfare. Absence of evidence does not mean evidence of absence, so we can’t make judgements based on this paper. This study only looks at one type of GM crop, so it wouldn’t be possible to extrapolate to other types even if we trusted the data.
Also, Roundup is a herbicide which you can easily buy for your garden. The significance of this is summarised in the words of Professor Ottoline Leyser, associate director of the Sainsbury Laboratory, University of Cambridge: “Like most of the GM debate, this work has very little to do with GM. The authors of the paper do not suggest that the effects are caused by genetic modification. They describe effects of the roundup herbicide itself and effects that they attribute to the activity of the enzyme introduced into the roundup resistant maize. There is good evidence that introducing genes into crops using GM techniques results in fewer changes to the crops than introducing them using conventional breeding.”
There’s lots more to be said, and I will keep saying it (and listening to it), but here are some ethical points not specifically related to the science, followed by the quotes.
Science in the media
I am not a toxicologist or statistician, so have to rely to a certain extent on other people’s views for that analysis, but I am a science press officer and I can say with certainty that Seralini acted in an unethical way with media relations. He sent out an embargoed press release to announce the imminent publication of his paper, as is normal, but made journalists sign confidentiality agreements for the paper, preventing them from consulting other experts before publication. A fundamental aspect of good science journalism is to seek expert opinions, and Seralini prevented this from happening. Carl Zimmer wrote a great article about this, explaining why journalists shouldn’t let themselves be played.
I heard an interesting podcast about the MMR scandal this week, and would recommend it to anyone who is looking for more insight into how science (especially criticised science) is communicated.
The retraction came after a letter was sent to Professor Seralini on 19th November 2013, stating “A more in-depth look at the raw data revealed that no definitive conclusions can be reached with this small sample size…” The letter was clear that it wasn’t accusing Seralini of fabricating data.
Unsurprisingly, there was a lot of criticism of the journal for retracting the paper, partly saying that inconclusiveness is not grounds for retraction. And they have a point, retraction is essential when results have been fabricated or when there are other problems with them (in my field of research there was a famous example of retraction when scientists realised that their results came about because the butterflies they were studying flew towards someone’s bright t shirt!). But inappropriate conclusions from legitimate data? That’s pretty common.
This post is far too long already, but the study has raised some publication ethics issues in my mind. Perhaps we need a more open peer review system? Currently scientists comment on papers anonymously. Were the letters to the editor criticising the paper good post-publication peer review, or do we need to make this a more standard and formal process, with specific guidelines for when such peer reviews give grounds for retraction? Could there be a way for Seralini’s data to be available, even if we disagree with his conclusions?
For me, a more valid reason for retraction is animal welfare issues. If a study wasn’t carried out in accordance with ethical standards, is it appropriate that it is published? And where do we draw the line? Does it make any difference if the study was sound in other ways? Or if the ethical violations are against people?
One criticism is that the journal has a Monsanto scientist on the board, and there are claims that this is why the paper was retracted. However, as you can see from links above and quotes below, criticism of the study also came from academic scientists, Government scientists and not for profit organisations. To say that they are all corrupt is a bold claim.
The paper is also very badly written, to the point that the abstract is almost unreadable (rats in the experimental group “…died 2–3 times more than controls…”, whatever ‘dying more’ might mean). While I very much sympathise with authors for whom English is not their native language, I don’t blame one correspondent with the journal for including criticism for the editor in his letter: “I was left wondering if anyone had really read the paper carefully.”
Quotes and reactions
Analysis of Seralini’s claims by the European Food Safety Authority (EFSA) concluded that “The study as reported by Séralini et al. was found to be inadequately designed, analysed and reported.”
The German Institute for Risk Assessment found that “A study of the University of Caen neither constitutes a reason for a re-evaluation of genetically modified NK603 maize nor does it affect the renewal of the glyphosate approval.”
Similarly, Food Standards Australia New Zealand said “On the basis of the many scientific deficiencies identified in the study, FSANZ does not accept the conclusions made by the authors and has therefore found no justification to reconsider the safety of NK603 corn, originally approved in 2002.”
Of course there are many who stand by Seralini’s findings, and believe that it is pressure from industry that has caused people to criticise the paper. I wanted to share quotes from people who have relevant expertise but do not work in a GM-related area, so I have no reason to believe they could possibly have anything to gain from the study.
Professor Alan Boobis, Professor of Biochemical Pharmacology at Imperial College London, said: “Retraction of a scientific article should always be seen as a last resort. Even when inconclusive, an article can contribute to scientific discourse on a topic. This is why replication of findings is so important. However, there are instances where the conclusions of a paper significantly over-interpret the findings, as was the case here. Whilst always of concern, this is particularly problematic when the subject of the paper is of considerable public and media interest. Hence, in the case of the article by Seralini et al I believe that the journal has acted responsibly and appropriately in evaluating all the data and taking this decision.”
Professor David Spiegelhalter, Winton Professor of the Public Understanding of Risk at the University of Cambridge, said: “It was clear from even a superficial reading that this paper was not fit for publication, and in this instance the peer review process did not work properly. But at least this has now been remedied and the journal has recognised that no conclusions can be drawn from this study, so I suppose it is better late than never. Sadly the withdrawal of this paper will not generate the publicity garnered by its initial publication.”
Seralini, GE (2013). Long term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize Food and Chemical Toxicology DOI: 10.1016/j.fct.2012.08.005