Climatic Research Unit email controversy

The Hubert Lamb Building, University of East Anglia, where the Climatic Research Unit is based

The Climatic Research Unit hacking incident came to light in November 2009 when it was discovered that thousands of e-mails and other documents had been obtained through the hacking of a server used by the Climatic Research Unit (CRU) of the University of East Anglia (UEA) in Norwich, England. The subsequent dissemination of the material caused a controversy, dubbed "Climategate", regarding whether or not the e-mails indicated misconduct by climate scientists. The UEA described the incident as an illegal taking of data. The police are conducting a criminal investigation of the server breach and subsequent personal threats made against some of the scientists mentioned in the e-mails.

The Information Commissioner's Office stated that the UEA had breached the Freedom of Information Act by not dealing properly with requests for information related to climate science research made by David Holland, a retired engineer, but as sanctions had to be imposed within six months of the offence it was too late to impose them.

The University of East Anglia has announced that an independent review of the allegations will be carried out by Sir Muir Russell and that the CRU's director, Professor Phil Jones, would stand aside from his post during the review.

Timeline

The incident began when a person or persons unknown accessed a server used by the Climatic Research Unit and copied 160 MB of data containing more than 1,000 e-mails and 3,000 other documents. The University of East Anglia stated that the server from which the data were taken was not one that could easily have been accessed and the data could not have been released inadvertently. It is not known when the breach occurred.

The breach was first discovered on 17 November 2009 after the server of the RealClimate website was hacked and a copy of the stolen data was uploaded. According to Gavin Schmidt of RealClimate, "At around 6.20am (EST) Nov 17th, somebody hacked into the RC server from an IP address associated with a computer somewhere in Turkey, disabled access from the legitimate users, and uploaded a file FOIA.zip to our server." A link to the file on the RealClimate server was posted from a Russian IP address to the Climate Audit blog at 7.24 am (EST i.e. at 2009-11-17 12:24Z) with the comment "A miracle just happened". Schmidt discovered the hack minutes after it occurred. He temporarily shut down the website and deleted the uploaded file. RealClimate reported that they had notified the University of East Anglia of the incident.

On 19 November an archive file containing the data was uploaded to a server in Tomsk, Russia before being copied to numerous locations across the Internet. An anonymous post from a Saudi Arabian IP address to the climate-sceptic blog The Air Vent, described the material as "a random selection of correspondence, code, and documents" and defended the hacking on the grounds that climate science is "too important to be kept under wraps".

The Norfolk police subsequently confirmed that they were "investigating criminal offences in relation to a data breach at the University of East Anglia" with the assistance of the Metropolitan Police's Central e-Crime unit, the Information Commissioner's Office (ICO) and the National Domestic Extremism Team (NDET). Commenting on the involvement of the NDET, a spokesman said: "At present we have two police officers assisting Norfolk with their investigation, and we have also provided computer forensic expertise. While this is not strictly a domestic extremism matter, as a national police unit we had the expertise and resource to assist with this investigation, as well as good background knowledge of climate change issues in relation to criminal investigations." However, the police cautioned that "major investigations of this nature are of necessity very detailed and as a consequence can take time to reach a conclusion."

On 27 January 2010, the ICO released a statement referring to requests made by an individual under the Freedom of Information Act: "The e-mails which are now public reveal that Mr Holland’s requests under the Freedom of Information Act were not dealt with as they should have been under the legislation. Section 77 of the Act makes it an offence for public authorities to act so as to prevent intentionally the disclosure of requested information." They said no legal action could be taken against the university because the six-month limit for prosecution set by the Act had expired.

Content of the documents

The material comprised more than 1,000 e-mails, 2,000 documents, as well as commented source code, pertaining to climate change research covering a period from 1996 until 2009. Some of the e-mails which have been widely publicised included discussions of how to combat the arguments of climate change sceptics, unflattering comments about sceptics, queries from journalists, and drafts of scientific papers. There have been assertions that these discussions indicated efforts to shut out dissenters and their points of view, and included discussions about destroying files in order to prevent them from being revealed under the UK Freedom of Information Act 2000. A review by the Associated Press of all the e-mails found that they did not support claims of faking of science, but did show disdain for sceptical critics. Scientists had discussed avoiding sharing information with critics, but the documents showed no evidence that any data was destroyed. Researchers also discussed in e-mails how information they had released on request was used by critics to make personal attacks on researchers. In an interview with The Guardian, Phil Jones said "Some of the emails probably had poorly chosen words and were sent in the heat of the moment, when I was frustrated. I do regret sending some of them. We've not deleted any emails or data here at CRU." He confirmed that the e-mails that had sparked the most controversy appeared to be genuine.

The quality of some of the source code included in the documents has been criticised, and an associated README file has been interpreted as suggesting that some data was simply made up. Myles Allen of the Climate Dynamics group at Oxford has said that the code under discussion is not that used in actual climate reconstructions, which is maintained elsewhere.

E-mails

Nearly all of the e-mails concerned technical and mundane aspects of climate research, such as data analysis and details of scientific conferences. The controversy has focused on a small number of e-mails, particularly those sent to or from climatologists Phil Jones, the head of the CRU, and Michael E. Mann of Pennsylvania State University (PSU), one of the originators of the graph of temperature trends dubbed the "hockey stick graph".

Officials from the Information Commissioner's Office have stated that the e-mails show that requests from a retired Northampton engineer under the Freedom of Information Act were "not dealt with as they should have been," and the rules were breached by withholding requested information. The UEA's vice chancellor, Edward Acton, said that the ICO's opinion was of "grave concern" to the university.

Jones e-mail of 16 Nov 1999

An excerpt from one November 1999 e-mail authored by Phil Jones, which the UEA has stated refers to a graph he was preparing as a diagram for the cover of the World Meteorological Organization (WMO) statement on the status of global climate in 1999:

"I've just completed Mike's Nature trick of adding in the real temps to each series for the last 20 years (ie, from 1981 onwards) and from 1961 for Keith's to hide the decline."

The graph showed three series of paleoclimate reconstructions, based on records of tree rings, corals, ice cores, lake sediments, etc., along with historical and instrumental records. "Mike's Nature trick" referred to a paper published by Michael Mann in Nature in 1998, which combined various proxy records with actual temperature records. Mann described the "trick" as simply a concise way of showing the two kinds of data together while still clearly indicating which was which. He said that there was nothing "hidden or inappropriate" about it, and that his method of combining proxy data had been corroborated by numerous statistical tests and matched thermometer readings taken over the past 150 years. A press release by the University of East Anglia said that the "trick" was using instrumental data to meet a requirement of showing temperatures more recent than those covered by the proxy based temperature reconstructions, and that the use of the word "trick" was not intended to imply any deception. An editorial in Nature said that 'trick' was slang for a clever (and legitimate) technique.

They had published a statement on the divergence problem in 1998, and had recommended that the post 1960 part of their reconstruction should not be used. Jones stated that the email was "written in haste" and that, far from seeking to hide the decline, CRU had published a number of articles on the problem. The implications of the decline are discussed in Chapter 6 of the 2007 IPCC Fourth Assessment Report, which describes discussion of various possible reasons for the divergence which does not affect all the trees, and says that there is no consensus about the cause. It notes that Briffa et al. specifically excluded the post 1960 data, which is therefore not shown in the graph of their reconstruction in the report. In his review comments on the report, Stephen McIntyre objected to this graph being truncated, and said that the whole reconstruction should be shown with comments to deal with the "divergence problem". The IPCC response was that this would be inappropriate.

Richard Lindzen of the Massachusetts Institute of Technology has gone on the record accusing Mann of data rigging and outright falsification. Other climatologists disputed Lindzen's comments. Thomas Peterson of the National Climatic Data Center said he had seen nothing in the emails that called the fundamental science into question, and Andrew Solow of Woods Hole Oceanographic Institution agreed that there was no trickery, saying he would use the word trick to describe some methodological step, but expressed the view that the basis of reconstructions had been unclear. Several scientific sources state that the decline being referred to is a decline in tree ring climate proxy metrics, not temperature. Andrew Watson, Royal Society Research Professor at the UEA, said that the scientists had drawn the line to follow the tree-ring reconstruction up to 1960 and the measured temperature after that."

Steve McIntyre said that the "trick to hide the decline" consisted of removing tree-ring data from the later half of the 20th century. He said that since the cause of the divergence problem is unknown, and it may have existed in earlier periods, tree ring records cannot be used to estimate temperatures in the past.

Before the incident, continuing research had already presented reconstructions based on more proxies, and found similar results with or without the tree ring records.

Mann e-mail of 11 Mar 2003

Main article: Soon and Baliunas controversy

In one e-mail, as a response to an e-mail indicating that a paper in the scientific journal Climate Research had questioned assertions that the 20th century was abnormally warm, Mann wrote:

"I think we have to stop considering Climate Research as a legitimate peer-reviewed journal. Perhaps we should encourage our colleagues in the climate research community to no longer submit to, or cite papers in, this journal."

Mann told the Wall Street Journal that he didn't feel there was anything wrong in saying "we shouldn't be publishing in a journal that's activist."

Mann was not alone in expressing concern about the peer review process of the journal. Half of the journal's editorial board, including editor-in-chief Hans von Storch, resigned in the wake of controversy surrounding the article's publication. The publisher later admitted that the paper's major findings could not "be concluded convincingly from the evidence provided in the paper. [Climate Research] should have requested appropriate revisions of the manuscript prior to publication."

Jones e-mail of 8 Jul 2004

An 8 July 2004 e-mail from Phil Jones to Michael Mann said in part:

"The other paper by MM is just garbage. [...] I can't see either of these papers being in the next IPCC report. Kevin and I will keep them out somehow — even if we have to redefine what the peer-review literature is!"

The IPCC has stated that its procedures mean there is "no possibility of exclusion of any contrarian views, if they have been published in established journals or other publications which are peer reviewed." In relation to the IPCC Fourth Assessment Report (AR4), the IPCC chairman Rajendra Pachauri stated that the papers that had been criticised "were actually discussed in detail in chapter six of the Working Group I report of the AR4. Furthermore, articles from the journal Climate Research, which was also decried in the emails, have been cited 47 times in the Working Group I report."

A Nature editorial stated that the UEA scientists had been sharply critical of the quality of the two papers, but "neither they nor the IPCC suppressed anything: when the assessment report was published in 2007 it referenced and discussed both papers." Peter Kelemen, a professor of geochemistry at Columbia University's Department of Earth and Environmental Sciences, said that "If scientists attempted to exclude critics' peer-reviewed papers from IPCC reports, this was unethical in my view." Rajendra Pachauri responded that the IPCC has "a very transparent, a very comprehensive process which ensures that even if someone wants to leave out a piece of peer reviewed literature there is virtually no possibility of that happening."

The independent review commissioned by the University of East Anglia will, inter alia, evaluate whether CRU's peer-review practices comply with best scientific practice.

Jones e-mail of 2 Feb 2005

A 2 February 2005 email from Phil Jones to Michael Mann includes:

"And don't leave stuff lying around on ftp sites - you never know who is trawling them. The two MMs have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I'll delete the file rather than send to anyone. Does your similar act in the US force you to respond to enquiries within 20 days?—our does! The UK works on precedents, so the first request will test it. We also have a data protection act, which I will hide behind.

Pro-Vice Chancellor of Research at University of East Anglia, Trevor Davies, said that no data was deleted or "otherwise dealt with in any fashion with the intent of preventing the disclosure". In response to allegations that CRU avoided obligations under the UK Freedom of Information Act, independent investigator Muir Russell plans to review CRU's "policies and practices regarding requests under the Freedom of Information Act".

Jones e-mail of May 2008

In one e-mail, Phil Jones writes to Michael Mann, with the subject line "IPCC & FOI"

"Can you delete any emails you may have had with Keith re AR4? Keith will do likewise…Can you also email Gene and get him to do the same? I don't have his new email address."

Critics say that the e-mails showed that scientists were conspiring to delete e-mails and documents to prevent them from being released. George Monbiot, a supporter of the scientific consensus, wrote that Jones' resignation is warranted on the basis of his statement in this email alone.

The UK Information Commissioner's Office (ICO) oversees the FOI process there, and issued the following statement:

"Destroying requested information outside of an organisation's normal policies is unlawful and may be a criminal offence if done to prevent disclosure."

Trevor Davies responded by saying that despite Jones' suggestion to delete records, no records were actually deleted.

Santer e-mail of 12 Nov 2009

In one e-mail, climate scientist Benjamin Santer of Lawrence Livermore National Laboratory commented on a request for data and correspondence from science blogger Stephen McIntyre under the Freedom of Information Act 2000 (FOI):

"My personal opinion is that both FOI requests [for data related to a 2008 paper and for correspondence dating back to 2006] are intrusive and unreasonable. Steven McIntyre provides absolutely no scientific justification or explanation for such requests. ... McIntyre has no interest in improving our scientific understanding of the nature and causes of climate change. He has no interest in rational scientific discourse....We should be able to conduct our scientific research without constant fear of an "audit" by Steven McIntyre; without having to weigh every word we write in every email we send to our scientific colleagues."

In an Associated Press interview, McIntyre disagreed with his portrayal in emails, and said "Everything that I've done in this, I've done in good faith."

FactCheck noted that the great majority of CRU's data is already freely available, and the scientists were reluctant to supply their own correspondence, code and data to people whose motives seemed questionable to them. It is not clear that any actual obstruction happened, and emails show the scientists discussing with university officials and lawyers their obligations under the new legislation, informing critics that data is already freely available, or that the information has been sent to them. This question is to form part of the East Anglia investigation.

Trenberth e-mail of 12 Oct 2009

An email written by Kevin Trenberth discussed gaps in understanding of recent temperature variations:

"The fact is that we can't account for the lack of warming at the moment and it is a travesty that we can't,"

Trenberth told the Associated Press that the email referred to an article he authored calling for improvement in measuring global warming to describe unusual data, such as rising sea surface temperatures. The word travesty refers to what Trenberth sees as an inadequate observing system that, were it more adequate, would be able to track the warming he believes is there.

In a statement on his NCAR webpage Trenberth states that,

"It is amazing to see this particular quote lambasted so often. It stems from a paper I published this year bemoaning our inability to effectively monitor the energy flows associated with short-term climate variability. It is quite clear from the paper that I was not questioning the link between anthropogenic greenhouse gas emissions and warming, or even suggesting that recent temperatures are unusual in the context of short-term natural variability."

Code and Documentation

The CRU files also included temperature data processing software written in Fortran and IDL as well as programmer comments and a readme file. On BBC Newsnight, software engineer John Graham-Cumming said that the code lacked clear documentation and an audit history, and included a bug in its error handling which, if it occurred, would ignore data without warning. Myles Allen, head of the Climate Dynamics group at the University of Oxford, said that the code investigated by Newsnight had nothing at all to do with the HadCRUT temperature record used for climate reconstructions, which is maintained at the Met Office and not at CRU.

Responses

In the United Kingdom and United States, there were calls for official inquiries into issues raised by the documents. Lord Lawson, a prominent British Conservative politician and founder of the Global Warming Policy Foundation said, "The integrity of the scientific evidence... has been called into question. And the reputation of British science has been seriously tarnished. A high-level independent inquiry must be set up without delay." United States Senator Jim Inhofe also planned to demand an inquiry.

University of East Anglia

The University of East Anglia was notified of the possible security breach on 17 November, but when the story was published in the press on 20 November they had no statement ready. On 24 November, Trevor Davies, the University of East Anglia pro-vice-chancellor with responsibility for research, rejected calls for Jones' resignation or firing: "We see no reason for Professor Jones to resign and, indeed, we would not accept his resignation. He is a valued and important scientist." The university announced it would conduct an independent review to "address the issue of data security, an assessment of how we responded to a deluge of Freedom of Information requests, and any other relevant issues which the independent reviewer advises should be addressed."

The university announced on 1 December that Phil Jones was to stand aside as director of the Unit until the completion of an independent review. Two days later, the university announced that Sir Muir Russell would chair the review, and would "examine e-mail exchanges to determine whether there is evidence of suppression or manipulation of data" as well as review CRU's policies and practices for "acquiring, assembling, subjecting to peer review, and disseminating data and research findings" and "their compliance or otherwise with best scientific practice". In addition, the investigation would review CRU's compliance with Freedom of Information Act requests and also 'make recommendations about the management, governance and security structures for CRU and the security, integrity and release of the data it holds."

UK Met Office

On November 23, a spokesman for the Met Office, a UK agency which works with the CRU in providing global-temperature information, said there was no need for an inquiry. "The bottom line is that temperatures continue to rise and humans are responsible for it. We have every confidence in the science and the various datasets we use. The peer-review process is as robust as it could possibly be."

On December 5, however, concerned that public confidence in the science had been damaged by leaked e-mails, the Met Office indicated their intention to re-examine 160 years of temperature data, as well as to release temperature records for over 1000 worldwide weather stations online. The Met Office remained confident that its analysis will be shown to be correct and that the data would show a temperature rise over the past 150 years.

UK Government

On January 22, 2010, the Science and Technology Committee of the Parliament of the United Kingdom announced it would conduct an inquiry into the incident, examining the implications of the disclosure for the integrity of scientific research, reviewing the scope of the independent Muir review announced by the UEA, and reviewing the independence of international climate data sets. The Committee plans to hold an oral evidence session in March 2010.

With reference to FOI requests made by David Holland, a retired engineer in Northampton, the Deputy Information Commissioner with responsibility for the Freedom of Information Act Graham Smith issued a statement on 27 January 2010 that "The emails which are now public reveal that Mr Holland's requests under the Freedom of Information Act were not dealt with as they should have been under the legislation. Section 77 of the Freedom of Information Act makes it an offence for public authorities to act so as to prevent intentionally the disclosure of requested information." He also said that as sanctions have to be imposed within six months of the offence it was too late to impose sanctions, but the ICO would be making a case for the law to be changed for future offences. He was advising the university of East Anglia on its legal obligations, and the ICO would be considering whether to take regulatory action once reports of the independent and police investigations were available.

Other responses

Rajendra Pachauri, chairman of the Intergovernmental Panel on Climate Change, told the BBC that he considered the affair to be "a serious issue and we will look into it in detail." He later clarified that the IPCC would review the incident to identify lessons to be learned, and he rejected suggestions that the IPCC itself should carry out an investigation. The only investigations being carried out were those of the University of East Anglia and the British police.

Pennsylvania State University announced it would review the work of Michael Mann, in particular looking at anything that had not already been addressed in an earlier National Academy of Sciences review which had found some faults with his methodology but agreed with the results. In response, Mann said he would welcome the review. As a result of the inquiry, the investigatory committee determined there was no credible evidence Mann suppressed or falsified data, destroyed email, information and/or data related to AR4, or misused privileged or confidential information. However, the committee was unsure if Mann operated within the accepted practices within the academic community for proposing, conducting, or reporting research or other scholarly activities, and a such, referred that charge to an investigatory committee of faculty members.

See also

• Global warming conspiracy theory
• Global warming controversy

External links

• "The Great Climategate Debate". A video of a lecture held at the MIT School of Science on December 10, 2009. The moderator was Henry D. Jacoby (MIT). Speakers were Kerry Emanuel (MIT), Judith Layzer (MIT), Stephen Ansolabehere (MIT and Harvard), Ronald G. Prinn (MIT), and Richard Lindzen (MIT).