Séralini affair: Difference between revisions

From Wikipedia, the free encyclopedia
Content deleted Content added
→‎top: let's simplify this. The lede is meant to lead the reader in, not choke them with dense information like a blipvert; also general fixes to sense & markup
Line 1: Line 1:
{{Use dmy dates|date=March 2015}}
{{Use dmy dates|date=March 2015}}


The '''Séralini affair''' is the name for the controversy about a particular experiment conducted by French [[Molecular biology|molecular biologist]] [[Gilles-Éric Séralini]]. Séralini fed [[Monsanto]]'s [[RoundUp]]-tolerant NK603 [[genetically modified maize]] (called corn in North America), as well as [[glyphosate]], to rats and published results which claimed that the corn and the herbicide were toxic{{cn|date=December 2015}}<!-- We need an exact source for this claim. --> to the animals in the journal ''[[Food and Chemical Toxicology]]''.
The '''Séralini affair''' is the name for the controversy surrounding publication of a [[journal article]] by French [[Molecular biology|molecular biologist]] [[Gilles-Éric Séralini]]. The article, which appeared in ''Food and Chemical Toxicology ''(FCT), claimed there was a strong link between [[genetically modified organism]]s and cancer, but was subsequently retracted because its conclusions were found not to be reliable.<ref name=qa/>


The group's conclusions and experimental design were heavily criticized, along with its publication strategy.<ref name="Arjo" /><ref name="Kuntz">{{cite journal |doi=10.1038/embor.2012.214 |title=Why the postmodern attitude towards science should be denounced |year=2013 |last1=Kuntz |first1=Marcel |journal=EMBO Reports |volume=14 |issue=2 |pages=114–6 |pmid=23306654 |pmc=3566841}}</ref> At the [[Science by press conference|press conference]] announcing the publication of the paper, Seralini displayed photographs of rats with large [[tumor]]s, and emphasized his cancer findings.<ref name="Arjo"/> The press conference received extensive coverage in the media; "within hours, the news had been blogged and tweeted more than 1.5 million times. At the press conference, Séralini also announced that he was releasing a book and a [[documentary film]] on the research. Séralini required journalists to sign a [[confidentiality agreement]] before viewing the study in advance of the conference.<ref name="Arjo">{{cite journal |doi= 10.1007/s11248-013-9692-9 |title= Plurality of opinion, scientific discourse and pseudoscience: An in depth analysis of the Séralini et al. Study claiming that Roundup™ Ready corn or the herbicide Roundup™ cause cancer in rats |year= 2013 |last1= Arjó |first1= Gemma |last2= Portero |first2= Manuel |last3= Piñol |first3= Carme |display-authors=3|last4= Viñas |first4= Juan |last5= Matias-Guiu |first5= Xavier |last6= Capell |first6= Teresa |last7= Bartholomaeus |first7= Andrew |last8= Parrott |first8= Wayne |last9= Christou |first9= Paul |journal= Transgenic Research |volume= 22 |issue= 2 |pages= 255–67 |pmid= 23430588}}</ref> The confidentiality agreement prohibited them from contacting other researchers in advance of the conference. The press conference and publication occurred a few weeks before the vote on [[California Proposition 37 (2012)]], which called for labelling [[genetically modified food]].
The publicity surrounding publication of the article attracted criticism.<ref name="Arjo" /><ref name="Kuntz">{{cite journal |doi=10.1038/embor.2012.214 |title=Why the postmodern attitude towards science should be denounced |year=2013 |last1=Kuntz |first1=Marcel |journal=EMBO Reports |volume=14 |issue=2 |pages=114–6 |pmid=23306654 |pmc=3566841}}</ref> At a [[Science by press conference|press conference]] announcing his findings, Séralini emphasized the cancer aspects of the work and displayed photographs of rats with large [[tumor]]s.<ref name="Arjo"/> The press conference received extensive coverage in the media and within hours, the news had been blogged and tweeted more than 1.5 million times. At the press conference, Séralini also announced that he was releasing a book and a [[documentary film]] on the research. Séralini required journalists to sign a [[confidentiality agreement]] before viewing the study in advance of the conference.<ref name="Arjo">{{cite journal |vauthors=Arjó G, Portero M, Piñol C, Viñas J, Matias-Guiu X, Capell T, Bartholomaeus A, Parrott W, Christou P |title=Plurality of opinion, scientific discourse and pseudoscience: an in depth analysis of the Séralini et al. study claiming that Roundup™ Ready corn or the herbicide Roundup™ cause cancer in rats |journal=Transgenic Res. |volume=22 |issue=2 |pages=255–67 |year=2013 |pmid=23430588 |doi=10.1007/s11248-013-9692-9 |url=}}</ref> The confidentiality agreement prohibited them from contacting other researchers in advance of the conference. The press conference and publication occurred a few weeks before the vote on [[California Proposition 37 (2012)]], which called for labelling [[genetically modified food]].


After the paper was published, scientists and regulatory agencies concluded that the conclusions of the paper to be invalid on the basis of the experimental design; each arm in the study had too few rats to obtain useful data in a lifetime study of [[Sprague-Dawley rat]]s, which get cancer at a high rate over their lifetime.<ref name="Ricroch" /> Other publicly funded long term studies uncovered no health issues.<ref name="Kuntz" /><ref name="Ricroch">{{cite journal |doi=10.1016/j.nbt.2012.12.001 |title=Assessment of GE food safety using '-omics' techniques and long-term animal feeding studies |year=2013 |last1=Ricroch |first1=Agnès E. |journal=New Biotechnology |volume=30 |issue=4 |pages=349–54 |pmid=23253614}}</ref>
After the paper was published, scientists and regulatory agencies concluded that the conclusions of the paper were invalid on the basis of the experimental design; each arm in the study had too few rats to obtain useful data in a lifetime study of [[Sprague-Dawley rat]]s, which get cancer at a high rate over their lifetime.<ref name="Ricroch" /> Other publicly funded long term studies uncovered no health issues.<ref name="Kuntz" /><ref name="Ricroch">{{cite journal |doi=10.1016/j.nbt.2012.12.001 |title=Assessment of GE food safety using '-omics' techniques and long-term animal feeding studies |year=2013 |last1=Ricroch |first1=Agnès E. |journal=New Biotechnology |volume=30 |issue=4 |pages=349–54 |pmid=23253614}}</ref>


In November 2013, ''Food and Chemical Toxicology ''(FCT), retracted Séralini's paper after the authors refused to withdraw it.<ref name=Elsevier2013>{{cite web|title=Elsevier Announces Article Retraction from Journal Food and Chemical Toxicology|url=http://www.elsevier.com/about/press-releases/research-and-journals/elsevier-announces-article-retraction-from-journal-food-and-chemical-toxicology|publisher=Elsevier|accessdate=2013-11-29}}</ref> In June 2014 Séralini republished the article in the journal ''[[Environmental Sciences Europe]]'', which did not conduct any further [[peer review]]. Reviewers checked only that the scientific content of the paper had not changed.<ref>{{cite news|last1=Cassasus|first1=Barbara|title=Paper claiming GM link with tumours republished|url=http://www.nature.com/news/paper-claiming-gm-link-with-tumours-republished-1.15463|accessdate=31 August 2015|publisher=Nature|date=25 June 2014}}</ref>
In November 2013 ''FCT'' imposed retraction Séralini's paper after the authors refused to withdraw it.<ref name=Elsevier2013>{{cite web|title=Elsevier Announces Article Retraction from Journal Food and Chemical Toxicology|url=http://www.elsevier.com/about/press-releases/research-and-journals/elsevier-announces-article-retraction-from-journal-food-and-chemical-toxicology|publisher=Elsevier|accessdate=2013-11-29}}</ref> In June 2014 Séralini republished the article in the journal ''[[Environmental Sciences Europe]]'', which did not conduct any further [[peer review]]&nnsp;&ndash; reviewers checked only that the scientific content of the paper had not changed.<ref>{{cite news|last1=Cassasus|first1=Barbara|title=Paper claiming GM link with tumours republished|url=http://www.nature.com/news/paper-claiming-gm-link-with-tumours-republished-1.15463|accessdate=31 August 2015|publisher=Nature|date=25 June 2014}}</ref>


{{toclimit|3}}
{{toclimit|3}}

Revision as of 07:27, 20 December 2015

The Séralini affair is the name for the controversy surrounding publication of a journal article by French molecular biologist Gilles-Éric Séralini. The article, which appeared in Food and Chemical Toxicology (FCT), claimed there was a strong link between genetically modified organisms and cancer, but was subsequently retracted because its conclusions were found not to be reliable.[1]

The publicity surrounding publication of the article attracted criticism.[2][3] At a press conference announcing his findings, Séralini emphasized the cancer aspects of the work and displayed photographs of rats with large tumors.[2] The press conference received extensive coverage in the media and within hours, the news had been blogged and tweeted more than 1.5 million times. At the press conference, Séralini also announced that he was releasing a book and a documentary film on the research. Séralini required journalists to sign a confidentiality agreement before viewing the study in advance of the conference.[2] The confidentiality agreement prohibited them from contacting other researchers in advance of the conference. The press conference and publication occurred a few weeks before the vote on California Proposition 37 (2012), which called for labelling genetically modified food.

After the paper was published, scientists and regulatory agencies concluded that the conclusions of the paper were invalid on the basis of the experimental design; each arm in the study had too few rats to obtain useful data in a lifetime study of Sprague-Dawley rats, which get cancer at a high rate over their lifetime.[4] Other publicly funded long term studies uncovered no health issues.[3][4]

In November 2013 FCT imposed retraction Séralini's paper after the authors refused to withdraw it.[5] In June 2014 Séralini republished the article in the journal Environmental Sciences Europe, which did not conduct any further peer review&nnsp;– reviewers checked only that the scientific content of the paper had not changed.[6]

Background

Gilles-Éric Séralini is a professor of molecular biology at the University of Caen in France. He is founder and president of the scientific advisory board of the Committee of Research and Independent Information on Genetic Engineering (CRIIGEN), which is publicly opposed to genetically modified food (GM food).[7][8][9] Séralini founded CRIIGEN because he judged that studies on GM food safety were inadequate, and questioned their acceptance.[7]

In 2004, Monsanto sought approval in Europe to introduce corn rootworm resistant (MON 863) Bt maize, which led to controversy over acceptance by regulatory bodies of industry-funded toxicity studies and over their study design. Séralini was on the committee that reviewed MON 863 for the French government.[10] He was a major figure in those controversies and continues to be a critic of toxicity study design.[11]

In 2004, the genetically modified organism (GMO) Panel of the European Food Safety Authority (EFSA) recommended the approval of MON 863.[12] Its report and referenced changes in blood cell parameters and in kidney weights of tested rats.[12] Greenpeace sued for release of the rat feeding studies that Monsanto had provided. Monsanto contested the suit in order to protect its trade secrets.[13] In June 2005 a German court ordered the release of the original study.[13][14] With the full study in hand, critics of GM foods, including Séralini and Jeffrey M. Smith, pointed to differences in kidney size and blood composition, suggesting that the observed differences, as well as the study designs, challenged the regulatory concept of substantial equivalence.[15]

Previous Séralini papers

Prior to 2012, Séralini had published other peer-reviewed papers that concluded there are health risks to GM foods. However, some members of the scientific community and food safety authorities questioned whether Séralini's data were sufficient to support his conclusions.

In 2007, Séralini and two other authors from University of Caen and the University of Rouen published a Greenpeace-funded study of these data.[16][17][18] The study concluded that MON 863 caused health problems in rats, including weight changes, triglyceride level increases in females, changes in urine composition in males, and reduced function or organ damage in the liver, kidney, adrenal glands, heart, and haematopoietic system.[16] The study further recommended that safety experiments continue beyond than 90 days, as chronic organ problems are rarely evident in that amount of time.[16] Greenpeace cited the study in a press release, in which it called for a MON 863 recall and a review of testing methods.[19]

The paper prompted EFSA to reexamine the MON 863 safety data. It asked EU countries for any new data about the strain, new opinions on the original Monsanto study and a technical meeting with the authors of the 2007 CRIIGEN paper. EFSA concluded that all blood chemistry and organ weight values fell within the normal range for control animals[20] and that the paper used incorrect statistical methods.[21] These conclusions were reported by Markos Kyprianou (European Commissioner for Health and Consumer Policy) to the European Parliament on 9 July 2010.[22] The French Commission du Génie Biomoléculaire (AFBV) also reached critical conclusions.[23] Food Standards Australia New Zealand reviewed the 2007 Séralini study and concluded that "...all of the statistical differences between rats fed MON 863 corn and control rats are attributable to normal biological variation."[24][25]

In 2009, the Séralini lab published another study (Séralini 2009), which re-analyzed toxicity data for NK 603 (glyphosate resistant), MON 810 and MON 863 strains.[26] The data included three rat feeding studies published by Monsanto scientists on MON 810 (Bt corn).[27][28][29] This study concluded that the three crops caused liver, kidney and heart damage in the rats.[26]

EFSA reviewed Séralini 2009 and concluded that the authors' claims were not supported by the data in their paper, that many of the statistical criticisms of the 2007 paper also applied to the 2009 paper and that it included no new information that would change the EFSA's conclusions.[30] The French High Council of Biotechnologies Scientific Committee[31] (HCB) reviewed Séralini 2009 and concluded that it "..presents no admissible scientific element likely to ascribe any haematological, hepatic or renal toxicity to the three re-analysed GMOs."[32] The HCB also questioned the authors' independence, noting that, in 2010, the Séralini web page still showed a 2008 Austrian anti-GM article that had been previously withdrawn by the authors as flawed. Food Standards Australia New Zealand concluded that the results from Séralini 2009 were due to chance alone.[33]

A 2011 review by the Séralini lab, which used 19 published animal feeding studies, as well as data from several animal feeding studies submitted for regulatory approval, continued to find that GM food had liver and kidney effects that were sex and dose dependent, and advocated for longer and more elaborate toxicology tests for regulatory approval.[11]

2012 study

On 19 September 2012, Séralini and his colleagues published a peer-reviewed paper titled "Long-term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize" in FCT. The research cost €3.2 million.[34] It was funded by and run with CRIIGEN. It involved a two-year study of NK603 fed to rats.[35]

The abstract stated, "The health effects of a Roundup-tolerant genetically modified maize (from 11% in the diet), cultivated with or without Roundup, and Roundup alone (from 0.1 ppb in water), were studied 2 years in rats. In females, all treated groups died 2–3 times more than controls, and more rapidly. This difference was visible in 3 male groups fed GMOs. All results were hormone and sex dependent, and the pathological profiles were comparable." The study used 200 Sprague-Dawley rats, 100 male and 100 female, and divided them into twenty groups with 10 rats each; ten experimental conditions were separately tested on male and female rats for two years.

Scientific evaluation

The study has been criticized by various regulatory authorities and scientists.

Many claimed that Séralini's conclusions were impossible to justify given the statistical power of the study. Sprague-Dawley rats have a lifespan of about two years and have a high risk of cancer over their lifespan (one study concluded that over eighty percent of males and over seventy percent of females developed cancer under normal conditions).[36][37][38] The Séralini experiment covered the normal lifespan of these rats. The longer an experiment continues, the more rats get cancer naturally, that makes it harder to separate statistical "noise" from the hypothetical signal. For the study to achieve such separation (statistical power), each control and test group would have to include sufficiently many subjects.[35] Organisation for Economic Co-operation and Development (OECD) guidelines recommend 20 rats for chemical-toxicity studies, and 50 rats for carcinogenicity studies.[39]: 5–6  In addition, if the survival of the rats is less than 50% at 104 weeks (which is likely for Sprague-Dawley rats) the recommended number of rats is 65.[35][37][38] The Séralini study had only ten per group.[35]

Tom Sanders from King's College London noted a lack of data on amount of food given, and on growth rates. Further noting that rats are susceptible to mammary tumors when food intake is not restricted. Sanders said, "The statistical methods are unconventional ... and it would appear the authors have gone on a statistical fishing trip."[40]

The Washington Post quoted Marion Nestle, Paulette Goddard professor in the Department of Nutrition, Food Studies and Public Health at New York University and food safety advocate: "'[I] can’t figure it out yet....It’s weirdly complicated and unclear on key issues: what the controls were fed, relative rates of tumors, why no dose relationship, what the mechanism might be. I can’t think of a biological reason why GMO corn should do this.....So even though I strongly support labeling, I’m skeptical of this study.'"[41] Likewise, Dan Charles, writing for NPR, noted that in the study, rats that ate 33% GM food developed fewer tumors than did those who ate 11% GM food, suggesting the absence of a dose response.[42] University of Calgary Professor Maurice Moloney publicly wondered why the paper contained so many pictures of treated rats with horrific tumors, but no pictures of control group rats.[43]

Many national food safety and regulatory agencies condemned the paper. The German Federal Institute for Risk Assessment VP Reiner Wittkowski said in a statement, ""The study shows both shortcomings in study design and in the presentation of the collected data. This means that the conclusions drawn by the authors are not supported by the available data."[44] A joint report by three Canadian regulatory agencies also "identified significant shortcomings in the study design, implementation and reporting."[45] Similar conclusions were reached by the French HCB[31] and the National Agency for Food Safety,[46] the Vlaams Instituut voor Biotechnologie,[47] the Technical University of Denmark,[48] Food Standards Australia New Zealand,[49] the Brazilian National Technical Commission on Biosafety,[50] and EFSA.[39] EFSA concluded:

The study as reported by Séralini et al. was found to be inadequately designed, analysed and reported...The study as described by Séralini et al. does not allow giving weight to their results and conclusions as published. Conclusions cannot be drawn on the difference in tumour incidence between treatment groups on the basis of the design, the analysis and the results as reported. Taking into consideration Member States’ assessments and the authors’ answer to critics, EFSA finds that the study as reported by Séralini et al. is of insufficient scientific quality for safety assessments.[39]

The European Federation of Biotechnology industry association, which counts Monsanto and other biotech firms among its members,[51] called for the paper to be retracted, calling its publication a "dangerous failure of the peer-review system."[35] Six French national academies (of Agriculture, Medicine, Pharmacy, Science, Technology and Veterinarians[52]) issued a joint statement – "an extremely rare event in French science"[9] – condemning the study and the journal that published it.[52] The joint statement dismissed the study as 'a scientific non-event'.[9] FCT, an Elsevier imprint, has a peer review process, and at least three scientists reviewed the paper prior to publication. The journal published a statement in their November 2012 issue, that "the Editors have encouraged those people with concerns to write formally to the Editor-in-Chief, so that their views can be publicly aired."

In March 2013 FCT published a letter[53] from Erio Barale-Thomas,[54] Principal Scientist of Johnson & Johnson Pharmaceutical Research and Development and the President of the Conseil d’Administration of The Société Française de Pathologie Toxicologique (SFPT, French Society of Toxicologic Pathology[55]). SFPT is "a non governmental/non profit organization formed by veterinarians, physicians, pharmacists and biologists specialized in veterinary and toxicologic pathology. Its aim is to promote knowledge in pathology, toxicology and laboratory animal sciences for safety studies of drugs, chemicals and food products, and the role of the pathologist in the study design and data interpretation."[53] The letter criticized the Seralini study on several fronts, and concluded: "However, given this study presents serious deficiencies in the protocol, the procedures and the interpretation of the results, the SFPT cannot support any of the scientific claims drawn by the authors, and any relevance for human risk assessment. This letter presents the consensus scientific opinion of the Conseil d’Administration of the SFPT."[53]

The Belgian Federal Minister of Public Health asked the Belgian Biosafety Advisory Council (BBAC) to evaluate the paper. The BBAC was asked to "inform the Minister whether this paper (i) contains new scientific information with regard to risks for human health of GM maize NK603 and (ii) whether this information triggers a revision of the current authorisation for commercialisation for food and feed use of this GM maize in the European Union (EU)."[56] The BBAC committee, whose members are drawn from the Belgian biotech Professoriat,[56] pointed out that "the long duration of this study is a positive aspect since most of the toxicity studies on GMOs are performed on shorter periods," and concluded that:

"Given the shortcomings identified by the experts regarding the experimental design, the statistical analysis, the interpretation of the results, the redaction of the article and the presentation of the results, the Biosafety Advisory Council concludes that this study does not contain new scientifically relevant elements that may lead to reconsider immediately the current authorisation for food and feed use of GM maize NK603. Considering the issues raised by the study (i.e. long term assessment), the Biosafety Advisory Council proposes EFSA urgently to study in depth the relevance of the actual guidelines and procedures. It can find inspiration in the GRACE project[57] to find useful information and new concerted ideas."[56]: 9 

The study was also criticized by the European Society of Toxicologic Pathology, which expressed shock at the way the rats in the study were treated and questioned whether the study was legal to perform under European law.[58]

Responses

With a few exceptions, the scientific community dismissed the Seralini study and called for a more rigorous peer-review system in scientific journals.[59]

Support

Séralini and supporters defended the study design, the interpretation of the results, and manner and content of the publication.[60][61][62]

Support for the study came from the European Network of Scientists for Social and Environmental Responsibility (ENSSER), of which CRIIGEN is a member.[63][64] A subsequent study published in 2013 by ENSSER concluded that EFSA (European Food Safety Authority) applied double standards in evaluation of feeding studies, criticized EFSA's applied criteria.[65]

An open letter in support of Seralini's article, signed by about 130 scientists, scholars, and activists, was published in Independent Science News, a project of the Bioscience Resource Project.[66]

Other Séralini supporters criticized the retraction of the study, concluding the response was a product of industry-driven campaign and regard this as a concerning example of industry interference in the scientific process.[67]

Public and government officials

At the time of the initial release, French Prime Minister Jean-Marc Ayrault said that, if the results are confirmed, the government would press for a Europe-wide ban on the maize and The European Commission instructed the EFSA in Parma, Italy, to assess the study.[68] In late September 2012, Russia temporarily suspended importing GM corn as a result of the study[69] and in November 2012, Kenya banned all GM crops.[70]

Media

The press conference led to wide media coverage, which "energized opponents of GM food, especially in Europe".[68] Le Nouvel Observateur covered the press conference in a story called, "Yes, GMOs are poisons!".[71]

Jon Entine in Forbes stated, "Seralini's research is anomalous. Previous peer-reviewed rat feeding studies using the same products (NK603 and Roundup) have not found any negative food safety impacts. The Japanese Department of Environmental Health and Toxicology released a 52-week feeding study of GM soybeans in 2007, finding "no apparent adverse effect in rats." In 2012, a team of scientists at the University of Nottingham School of Biosciences released a review of 12 long-term studies (up to two years) and 12 multi-generational studies (up to 5 generations) of GM foods, concluding there is no evidence of health hazards."[72] Andrew Revkin wrote in a blog the study was another instance of "single-study syndrome", and that the study was in support of an "agenda".[73]

Henry I. Miller, in an opinion piece for Forbes, said "[Seralini] has crossed the line from merely performing and reporting flawed experiments to committing gross scientific misconduct and attempting fraud."[74] Séralini responded by saying, "...that he won't make any data available to the EFSA and the BfR until the EFSA makes public all the data under-pinning its 2003 approval of NK603 maize for human consumption and animal feed."[35]

The Guardian's Environmental Blog stated that the study linking GM maize to cancer "must be taken seriously by regulators" and that although it "attracted a torrent of abuse", "it cannot be swept under the carpet". They also noted CRIIGEN's funding of the research and reported Séralini's response: namely, that studies in support of GM food are usually funded by "corporates or by pro-biotech institutions".[34] Proponents of California's GM labeling referendum, Proposition 37, hailed the study.[75][76]

A statement about the controversy, and especially the attacks on Seralini, was published in Le Monde, signed by 140 French scientists; the letter said: "...the protocol followed in this study presents defects that are subject to debate within the scientific community.... We are deeply shocked by the image of our community that this controversy gives citizens. The risk expertise to human health or the environment is a difficult activity which is facing many uncertainties Many of the threats to our planet have been revealed by scientists isolated and confirmed by many studies coming from the scientific community. In this case, it would be more efficient to implement research on the health and environmental risks of GMOs and pesticides, improve toxicological protocols used for placing on the market and finance a variety of researchers in this domain...."[77]

Publication strategy

Séralini held a press conference on the day the study was released in which he "promoted the cancer results as the study’s major finding"[35] and which featured large images of rats with tumors.[2][78] At the press conference he also announced the release of a book and film about the study. Selected journalists were given early access to the paper on the condition that they sign a confidentiality agreement that prevented them from discussing the paper with other scientists before the embargo expired.[35] In contrast, embargo guidelines by journals such as Nature allow such disclosures.[79]

This approach was widely criticized. The original Agence France-Presse story noted: "Breaking with a long tradition in scientific journalism, the authors allowed a selected group of reporters to have access to the paper, provided they signed confidentiality agreements that prevented them from consulting other experts about the research before publication."[80][81]

A Nature editorial noted: "With such strong claims and the predictably large effect they will have on public opinion, researchers should take care how they present their findings to the public and the media. They should spell out their results clearly; emphasize the limitations and caveats; and make it clear that the data still need to be assessed, and replicated, by the scientific community. That didn't happen. The paper was promoted in a public-relations offensive, with a related book and film set for release this week. Furthermore, journalists wishing to report the research had to sign confidentiality agreements that prevented them from contacting other scientists for comment on the paper until after the embargo had expired. Some, to their credit, refused, or accepted and then revisited the story critically once their hands were no longer tied by these outrageous restrictions. The result was the exclusion of critical comment in many of the breaking stories — the ones that most people will remember."[82][83][84]

NPR program On the Media discussed the issue on 28 September 2012, with science journalist Carl Zimmer, who criticized science journalists who participated, as well as criticizing the Séralini lab.[85] Zimmer had earlier posted on his blog at Discover magazine, "This is a rancid, corrupt way to report about science."[86] Cosmos Magazine's Elizabeth Finkel, wrote, "...a clause barring the gathering of independent opinions is extraordinary. What it meant was that Séralini’s story, when it broke, got to prance unfettered in the media limelight before second opinions could dull its shine. By the time the storm of criticism blew in, the media limelight had moved on."[87] The ethics committee of the French National Centre for Scientific Research (CNRS) also criticized the public-relations offensive as "inappropriate for a high-quality and objective scientific debate, and reminded researchers working on controversial topics of the need to report results responsibly to the public."[35]

Retraction and republication

In November 2013, Elsevier announced that FCT was retracting the paper, after the authors refused to withdraw it.[5][88] The journal's editors concluded that while there was "no evidence of fraud or intentional misrepresentation of the data", the results were inconclusive and "[did] not reach the threshold of publication for Food and Chemical Toxicology". After an in-depth look at the study's raw data, no definitive conclusions could be reached regarding the role of either NK603 or glyphosate in overall mortality or tumor rates, given the high incidence of tumors in Sprague-Dawley rats and the small sample size. Normal variance could not be excluded as the cause of the results.[5] Following many enquiries about the retraction, FCT's editor-in-chief said that:

The retraction statement could have been clearer, and should have referred to the relevant COPE guidelines. The data are inconclusive, therefore the claim (i.e., conclusion) that Roundup Ready maize NK603 and/or the Roundup herbicide have a link to cancer is unreliable. Dr. Séralini deserves the benefit of the doubt that this unreliable conclusion was reached in honest error. The review of the data made it clear that there was no misconduct. However, to be very clear, it is the entire paper, with the claim that there is a definitive link between GMO and cancer that is being retracted. Dr. Séralini has been very vocal that he believes his conclusions are correct. In our analysis, his conclusions cannot be claimed from the data presented in this article.

— Wallace Hayes A, Editor in Chief of Food and Chemical Toxicology answers questions on retraction[1]

Séralini and his supporters strongly objected to the retraction,[88][89][90] and Séralini himself threatened to sue FCT.[91]

In June 2014, the original study was republished in the journal Environmental Sciences Europe.[92][93] The editor said that the paper was republished without further scientific peer review, "because this had already been conducted by Food and Chemical Toxicology, and had concluded there had been no fraud nor misrepresentation".[93] The republication renewed the controversy, but now with additional controversy over the behavior of the editors of both journals.[94]

In July 2015 the International Agency for Research on Cancer published a monograph on glyphosate, which contained an evaluation of the Séralini paper as republished in June 2014 and the conclusion, that the study “was inadequate for evaluation because the number of animals per group was small, the histopathological description of tumours was poor, and incidences of tumours for individual animals were not provided.” [95]

PLOS ONE study

Séralini responded to criticisms of his methodology (and specifically a lack of difference between rodent groups at higher doses) with a July 2015 paper in PLOS ONE claiming that all laboratory rodent diets are contaminated with "dangerous" levels of GMOs.[96] This has been strongly criticised by numerous experts,[97] for example, Tamara Galloway said that the study "speculates beyond the evidence presented in this paper".[98]

Lawsuits

In 2010, Séralini sued University of Paris VII Professor Marc Fellous, president of the French Association of Plant Biotechnology and the Association, for libel, claiming that they had unjustly criticized his scientific ability and his research because of its funder, Greenpeace. The judge ruled that because Fellous and other critics had financial ties to the agricultural biotechnology industry, their charge about the Greenpeace funding was defamatory, but refused to rule on the scientific matter. Fellous was fined 1000 euros. Séralini was awarded a symbolic 1 euro in damages and court costs.[99]

In 2012 Seralini sued for defamation. French Television reported, that after 3 years of deliberation the High Court of Paris had found the journal Marianne and journalist Jean-Claude Jaillette guilty on 6 November 2015. It also reported that the only prior accusation of fraud had been by Henry I. Miller in his Forbes opinion piece.[100]

See also

References

  1. ^ a b Wallace Hayes A (2014). "Editor in Chief of Food and Chemical Toxicology answers questions on retraction". Food Chem. Toxicol. 65: 394–5. doi:10.1016/j.fct.2014.01.006. PMID 24407018.
  2. ^ a b c d Arjó G, Portero M, Piñol C, Viñas J, Matias-Guiu X, Capell T, Bartholomaeus A, Parrott W, Christou P (2013). "Plurality of opinion, scientific discourse and pseudoscience: an in depth analysis of the Séralini et al. study claiming that Roundup™ Ready corn or the herbicide Roundup™ cause cancer in rats". Transgenic Res. 22 (2): 255–67. doi:10.1007/s11248-013-9692-9. PMID 23430588.
  3. ^ a b Kuntz, Marcel (2013). "Why the postmodern attitude towards science should be denounced". EMBO Reports. 14 (2): 114–6. doi:10.1038/embor.2012.214. PMC 3566841. PMID 23306654.
  4. ^ a b Ricroch, Agnès E. (2013). "Assessment of GE food safety using '-omics' techniques and long-term animal feeding studies". New Biotechnology. 30 (4): 349–54. doi:10.1016/j.nbt.2012.12.001. PMID 23253614.
  5. ^ a b c "Elsevier Announces Article Retraction from Journal Food and Chemical Toxicology". Elsevier. Retrieved 29 November 2013.
  6. ^ Cassasus, Barbara (25 June 2014). "Paper claiming GM link with tumours republished". Nature. Retrieved 31 August 2015.
  7. ^ a b "HH", CRIIGEN, 12 November 12, 2008 Profile, Pr Gilles Eric Séralini – President of the Scientific Board – Molecular Biology Professor
  8. ^ Carman, Tim (19 September 2012). "French scientists question safety of GM corn". Washington Post. Retrieved 20 May 2013.
  9. ^ a b c Agence France-Presse. 19 October 2012, as posted on phys.org. Six French academies dismiss study linking GM corn to cancer (Update 2)
  10. ^ "Comité de Recherche et d'Information Indépendantes sur le génie Génétique". CRIIGEN.
  11. ^ a b Séralini, Gilles-Eric; Mesnage, Robin; Clair, Emilie; Gress, Steeve; De Vendômois, Joël; Cellier, Dominique (2011). "Genetically modified crops safety assessments: Present limits and possible improvements". Environmental Sciences Europe. 23: 10. doi:10.1186/2190-4715-23-10.{{cite journal}}: CS1 maint: unflagged free DOI (link)[non-primary source needed]
  12. ^ a b EFSA Panel on Genetically Modified Organisms (2004). "Opinion of the Scientific Panel on genetically modified organisms [GMO] on a request from the Commission related to the safety of foods and food ingredients derived from insect-protected genetically modified maize MON 863 and MON 863 x MON 810, for which a request for placing on the market was submitted under Article 4 of the Novel Food Regulation (EC) No 258/97 by Monsanto". The EFSA Journal. 50: 1–25. doi:10.2903/j.efsa.2004.50.
  13. ^ a b "MON863 maize: Court orders disclosure of all documents. No secrets in safety matters". GMO-Safety.eu.
  14. ^ Monsanto, 2002. 13-Week Dietary Subchronic Comparison Study with MON 863 Corn in Rats Preceded by a 1-Week Baseline Food Consumption Determination with PMI Certified Rodent Diet #5002 Monsanto full rat study
  15. ^ Jeffrey M. Smith January 2008. Study reveals GM threats Biophile Magazine, Issue 6.
  16. ^ a b c Séralini, Gilles-Eric; Cellier, Dominique; De Vendomois, Joël Spiroux (2007). "New Analysis of a Rat Feeding Study with a Genetically Modified Maize Reveals Signs of Hepatorenal Toxicity". Archives of Environmental Contamination and Toxicology. 52 (4): 596–602. doi:10.1007/s00244-006-0149-5. PMID 17356802.
  17. ^ "GM maize MON863: French scientists doubt safety". GMO Compass. 16 March 2007. Retrieved 11 November 2010.
  18. ^ Ananda, Rady (2010). "Three Approved GMOs Linked to Organ Damage" (PDF). Z Magazine. 23 (3). Retrieved 21 July 2010. The data 'clearly underlines adverse impacts on kidneys and liver, the dietary detoxifying organs, as well as different levels of damages to heart, adrenal glands, spleen, and haematopoietic system,' reported Gilles-Eric Séralini, a molecular biologist at Caen University.
  19. ^ "Regulatory systems for GE crops a failure: the case of MON863" (PDF). Greenpeace. Archived from the original (PDF) on 30 March 2009. Retrieved 21 July 2010. Greenpeace demands an immediate and complete recall of MON863 from the global market. We also call upon governments to undertake an urgent reassessment of all other authorised GE products and a strict review of current testing methods. {{cite web}}: Unknown parameter |deadurl= ignored (|url-status= suggested) (help)
  20. ^ Statement of the Scientific Panel on Genetically Modified Organisms on the analysis of data from a 90-day rat feeding study with MON 863 maize [1]
  21. ^ European Food Safety Authority (2007). "EFSA review of statistical analyses conducted for the assessment of the MON 863 90-day rat feeding study". doi:10.2903/j.efsa.2007.19r.
  22. ^ Les experts européens innocentent un OGM Le Figaro, 13 July 2007. Retrieved 27 October 2010
  23. ^ Les Organismes Génétiquement Modifiés, Annexe B. Avis de la commission du génie biomoléculaire sur l’étude statistique du CRIIGEN du maïs MON863 Report prepared for the French Prime Minister by the Centre d'Analyse Strategique, 20 July 2007. Retrieved 31 May 2013
  24. ^ "Review of the report by Séralini et al., (2007): "New analysis of a rat feeding study with a genetically modified maize reveals signs of hepatorenal toxicity"". FSANZ final assessment report. Retrieved 11 November 2010.[dead link]
  25. ^ "FSANZ reaffirms its risk assessment of genetically modified corn MON 863". FSANZ fact sheets 2007. 25 July 2010. Retrieved 11 November 2010.
  26. ^ a b De Vendômois, Joël Spiroux (2009). "A Comparison of the Effects of Three GM Corn Varieties on Mammalian Health". International Journal of Biological Sciences. 5 (7): 706–26. doi:10.7150/ijbs.5.706. PMC 2793308. PMID 20011136.
  27. ^ Hammond, B; Dudek, R; Lemen, J; Nemeth, M (2004). "Results of a 13 week safety assurance study with rats fed grain from glyphosate tolerant corn". Food and Chemical Toxicology. 42 (6): 1003–14. doi:10.1016/j.fct.2004.02.013. PMID 15110110.
  28. ^ Hammond, B.; Lemen, J.; Dudek, R.; Ward, D.; Jiang, C.; Nemeth, M.; Burns, J. (2006). "Results of a 90-day safety assurance study with rats fed grain from corn rootworm-protected corn". Food and Chemical Toxicology. 44 (2): 147–60. doi:10.1016/j.fct.2005.06.008. PMID 16084637.
  29. ^ Hammond, B.G.; Dudek, R.; Lemen, J.K.; Nemeth, M.A. (2006). "Results of a 90-day safety assurance study with rats fed grain from corn borer-protected corn". Food and Chemical Toxicology. 44 (7): 1092–9. doi:10.1016/j.fct.2006.01.003. PMID 16487643.
  30. ^ "EFSA Minutes of the 55th Plenary Meeting of the Scientific Panel on Genetically Modified Organisms Held on 27–28 January 2010 IN Parma, Italy, Annex 1, Vendemois et al 2009" (PDF). European Food Safety Authority report. Retrieved 11 November 2010.
  31. ^ a b French wiki: Haut conseil de biotechnologies[self-published source?]
  32. ^ "Opinion relating to the deposition of 15 December 2009 by the Member of Parliament, François Grosdidier, as to the conclusions of the study entitled "A comparison of the effects of three GM corn varieties on mammalian health"". English translation of French High Council of Biotechnologies Scientific Committee document. Retrieved 11 November 2010.
  33. ^ "Feeding studies and GM corn MON863". Food Standards Australia New Zealand. July 2012. Retrieved 10 October 2012.
  34. ^ a b Vidal, John (28 September 2012). "Study linking GM maize to cancer must be taken seriously by regulators". The Guardian. London. Retrieved 8 May 2013.
  35. ^ a b c d e f g h i Butler, Declan (2012). "Hyped GM maize study faces growing scrutiny". Nature. 490 (7419): 158. doi:10.1038/490158a. PMID 23060167.
  36. ^ Suzuki, H; Mohr, U; Kimmerle, G (1979). "Spontaneous endocrine tumors in Sprague-Dawley rats". Journal of cancer research and clinical oncology. 95 (2): 187–96. doi:10.1007/BF00401012. PMID 521452.
  37. ^ a b "Mortality and In-Life Patterns in Sprague-Dawley" (PDF). Huntingdon Life Sciences. Retrieved 26 October 2012.
  38. ^ a b "Sprague Dawley" (PDF). Harlan. Retrieved 26 October 2012.
  39. ^ a b c European Food Safety Authority (2012). "Final review of the Séralini et al. (2012a) publication on a 2-year rodent feeding study with glyphosate formulations and GM maize NK603 as published online on 19 September 2012 in Food and Chemical Toxicology". EFSA Journal. 10 (11): 2986. doi:10.2903/j.efsa.2012.2910. {{cite journal}}: Unknown parameter |laydate= ignored (help); Unknown parameter |laysource= ignored (help); Unknown parameter |laysummary= ignored (help)
  40. ^ Ben Hirschler and Kate Kelland. Reuters. September 20, 2012 Study on Monsanto GM corn concerns draws skepticism
  41. ^ Tim Carman for the Washington Post. Posted at 07:30 PM ET, 19 September 2012. French scientists question safety of GM corn [2]
  42. ^ Charles, Dan (19 September 2012). "As Scientists Question New Rat Study, GMO Debate Rages On". NPR. Retrieved 1 November 2014.
  43. ^ Amos, Jonathan (19 September 2012). "French GM-fed rat study triggers furore". BBC News. Retrieved 22 August 2013.
  44. ^ Staff (1 October 2012) A study of the University of Caen neither constitutes a reason for a re-evaluation of genetically modified NK603 maize nor does it affect the renewal of the glyphosate approval German Federal Institute for Risk Assessment (BfR). Retrieved 14 October 2012
  45. ^ Staff, Food Directorate, Health Products and Food Branch, Health Canada; Animal Feed Division, Animal Health Directorate, Canadian Food Inspection Agency; Pest Management Regulatory Agency, Health Canada. October 25, 2012 Health Canada and Canadian Food Inspection Agency statement on the Séralini et al. (2012) publication on a 2-year rodent feeding study with glyphosate formulations and GM maize NK603
  46. ^ Staff (22 October 2012) French panel rejects study linking GM corn to cancer Agence France Presse. Retrieved 23 October 2012
  47. ^ Staff (8 October 2012) VIB concludes that Séralini study is not substantiated VIB Life Sciences Research Institute, Belgium. Retrieved 14 October 2012
  48. ^ Staff (October 2012) The Technical University of Denmark National Food Institute's assessment of a new long-term trial with genetically modified maize NK603 and spray Roundup (In Danish) Technical University of Denmark, Danish National Food Institute, Rertrieved 23 October 2012
  49. ^ "Response to Séralini paper on the long term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize". FSANZ website. October 2012. Archived from the original on 20 October 2012. Retrieved 13 November 2013.
  50. ^ "Ministry of Science Technology and Innovation : Considered Opinion" (PDF). Cibiogem.gob.mx. Retrieved 20 August 2013.
  51. ^ EFB (2013) Members
  52. ^ a b Avis des Académies nationales d’Agriculture, de Médecine, de Pharmacie, des Sciences, des Technologies, et Vétérinaire sur la publication récente de G.E. Séralini et al. sur la toxicité d’un OGM Communiqué de presse 19 octobre 2012
  53. ^ a b c Barale-Thomas, Erio (2013). "Letter to the editor". Food and Chemical Toxicology. 53: 473–4. doi:10.1016/j.fct.2012.10.041. PMID 23165156.
  54. ^ Erio Barale-Thomas linked in page
  55. ^ webpage of the Société Française de Pathologie Toxicologique
  56. ^ a b c Bioveiligheidsraad Conseil de Biosécurité (Belgian Biosafety Advisory Council) (2012). Advice of the Belgian Biosafety Advisory Council on the article by Séralini et al., 2012 on toxicity of GM maize NK603
  57. ^ "GRACE project". grace-fp7.eu.
  58. ^ Bartholomaeus, A; Parrott, W; Bondy, G; Walker, K; ILSI International Food Biotechnology Committee Task Force on Use of Mammalian Toxicology Studies in Safety Assessment of GM, Foods (November 2013). "The use of whole food animal studies in the safety assessment of genetically modified crops: limitations and recommendations". Critical reviews in toxicology. 43 Suppl 2: 1–24. doi:10.3109/10408444.2013.842955. PMID 24164514.
  59. ^ Martinelli, L; Karbarz, M; Siipi, H (February 2013). "Science, safety, and trust: the case of transgenic food". Croatian Medical Journal. 54 (1): 91–6. doi:10.3325/cmj.2013.54.91. PMID 23444254.
  60. ^ Séralini, Gilles-Eric; Mesnage, Robin; Defarge, Nicolas; Gress, Steeve; Hennequin, Didier; Clair, Emilie; Malatesta, Manuela; De Vendômois, Joël Spiroux (2013). "Answers to critics: Why there is a long term toxicity due to a Roundup-tolerant genetically modified maize and to a Roundup herbicide". Food and Chemical Toxicology. 53: 476–83. doi:10.1016/j.fct.2012.11.007. PMID 23146697.
  61. ^ "Letter to the Committee On Publication Ethics (COPE)" (PDF). 31 January 2014. Retrieved 28 June 2014. {{cite web}}: Cite uses deprecated parameter |authors= (help)
  62. ^ "GMOSeralini". Retrieved 24 April 2014.
  63. ^ Staff, ENSSER. Page last modified: 10-18-2012. ENSSER Comments on Séralini et al. 2012
  64. ^ European Network of Scientists for Social and Environmental Responsibility (ENSSER) (2012). Questionable biosafety of GMOs, double standards and, once again, a "shooting-the-messenger" style debate
  65. ^ Hartmut Meyer and Angelika Hilbeck (2013). "Rat feeding studies with genetically modified maize – a comparative evaluation of applied methods and risk assessment standards". Environmental Sciences Europe. doi:10.1186/2190-4715-25-33.{{cite journal}}: CS1 maint: unflagged free DOI (link)
  66. ^ Independent Science News (2012) Seralini and Science: an Open Letter, Oct 2 2012
  67. ^ Fagan, John; Traavik, Terje; Bøhn, Thomas (29 August 2015). "The Seralini affair: degeneration of Science to Re-Science?". Environmental Sciences Europe. 27 (19). doi:10.1186/s12302-015-0049-2. Retrieved 3 September 2015.{{cite journal}}: CS1 maint: unflagged free DOI (link)
  68. ^ a b Declan Butler for Nature News. September 25, 2012 Rat study sparks GM furore
  69. ^ Staff, Phys.org. September 26, 2012 Russia suspends Monsanto corn imports
  70. ^ Emily Willingham for Forbes. December 9, 2012. Seralini Paper Influences Kenya Ban of GMO Imports
  71. ^ "EXCLUSIF. Oui, les OGM sont des poisons ! – Le Nouvel Observateur". Tempsreel.nouvelobs.com. Retrieved 20 August 2013.
  72. ^ Entine, Jon (30 September 2012). "Does the Seralini Corn Study Fiasco Mark a Turning Point in the Debate Over GM Food?". Forbes.com. Retrieved 11 May 2013.
  73. ^ Andrew C. Revkin Single-Study Syndrome and the G.M.O. Food Fight September 20, 2012,dotearth.blogs.nytimes
  74. ^ Miller, Henry I. (25 September 2012). "Scientists Smell A Rat In Fraudulent Genetic Engineering Study" (opinion). Forbes.
  75. ^ Staff, Right to Know.Posts tagged Seralini
  76. ^ Pollack, Andrew (20 September 2012). "Foes of Modified Corn Find Support in a Study". New York Times. Retrieved 15 November 2014.
  77. ^ Andalo C, Chercheuse AHS, Atlan A, Auclair D, Austerlitz F, Barot S. Science et conscience English translation via Google Translate: Science and conscience. Le Monde. 14 November 2012.
  78. ^ CBS News. November 29, 2013 Journal retracts genetically modified corn study that found tumor risk in rats
  79. ^ "Embargoes on the web". nature.com.
  80. ^ Thomas Lumley for Stats Chat website. 20 September 2012 Roundup scare
  81. ^ Agence France-Presse 19th September 2012 France orders probe after rat study links GM to cancer
  82. ^ "Poison postures". Nature. 489 (7417): 474. September 2012. doi:10.1038/489474a. PMID 23025010.
  83. ^ Séralini, Gilles-Eric (2012). Tous Cobayes !: OGM, pesticides et produits chimiques. Editions Flammarion. ISBN 978-2-08-126236-2.
  84. ^ "Tous cobayes? (2012) – IMDb". IMDB. IMDB.com.
  85. ^ On the Media Radio Show, 2012 Sep 28 Manipulating Science Reporting
  86. ^ Carl Zimmer on Discover Magazine blog, The Loom. 21 September 2012 From Darwinius to GMOs: Journalists Should Not Let Themselves Be Played
  87. ^ GM Corn and Cancer: the Seralini Affair
  88. ^ a b Barbara Casassus for Nature News. November 28, 2013 Study linking GM maize to rat tumours is retracted: Publisher withdraws paper over authors' objections, citing weak evidence
  89. ^ Michael Hiltzik for the Los Angeles Times. November 29, 2013 Notorious anti-GMO study is retracted – creating more controversy
  90. ^ Portier, Christopher J.; Goldman, Lynn R.; Goldstein, Bernard D. (1 February 2014). "Inconclusive Findings: Now You See Them, Now You Don't!". Environmental Health Perspectives. 122 (2): A36–A36. doi:10.1289/ehp.1408106.
  91. ^ Entine, Jon (29 November 2013). "Séralini Threatens Lawsuit In Wake Of Retraction Of Infamous GMO Cancer Rat Study". Forbes. Retrieved 19 February 2014.
  92. ^ Séralini, Gilles-Eric; Clair, Emilie; Mesnage, Robin; Gress, Steeve; Defarge, Nicolas; Malatesta, Manuela; Hennequin, Didier; de Vendômois, Joël. "Republished study: long-term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize". Environmental Sciences Europe. 26 (1): 14. doi:10.1186/s12302-014-0014-5.{{cite journal}}: CS1 maint: unflagged free DOI (link)
  93. ^ a b Casassus, Barbara (26 June 2014). "Paper claiming GM link with tumours republished". Nature. Retrieved 26 June 2014.
  94. ^ Science Media Centre. June 25th, 2014. Controversial GM study republished – experts respond
  95. ^ IARC monograph on glyphosate, p. 35, right column
  96. ^ Mesnage, R; Defarge, N; Rocque, LM; Spiroux de Vendômois, J; Séralini, GE (2015). "Laboratory Rodent Diets Contain Toxic Levels of Environmental Contaminants: Implications for Regulatory Tests". PLoS One. 10 (7): e0128429. doi:10.1371/journal.pone.0128429. PMID 26133768.{{cite journal}}: CS1 maint: unflagged free DOI (link)
  97. ^ "expert reaction to study investigating content of and contaminants in laboratory rodent diets". Science Media Center. 2 July 2015. Retrieved 9 September 2015.
  98. ^ "Pesticides in lab rat feed causes a stir". news24.com. 3 July 2015. Retrieved 9 September 2015.
  99. ^ Vincent Olivier for L'Express. January 19, 2011. OGM: deux chercheurs au tribunal English translation
  100. ^ Elise Ferret (27 November 2015). "Le professeur Gilles-Eric Séralini remporte le procès en diffamation face à Marianne". France 3, Basse-Normandie. France Télévisions. Retrieved 1 December 2015.

External links