Wegman Report

The Wegman Report (officially called the Ad Hoc Committee Report on the 'Hockey Stick' Global Climate Reconstruction) was prepared in 2006 by three statisticians led by Edward Wegman at the request of Rep. Joe Barton of the United States House Committee on Energy and Commerce to validate criticisms made by Stephen McIntyre and Ross McKitrick of reconstructions of the temperature record of the past 1000 years, in particular the reconstructions by Mann, Bradley and Hughes of what had been dubbed the hockey stick graph.

Background

Investigations of paleoclimate date back to the 1930s, but quantitative methods were slow to come into use. In the 1960s, Hubert Lamb generalised from historical documents and temperature records of central England to propose a Medieval Warm Period from around 900 to 1300, followed by Little Ice Age. This was the basis of a "schematic diagram" featured in the IPCC First Assessment Report beside cautions that the medieval warming might not have been global. The use of proxy indicators to get quantitative estimates of the temperature record of past centuries was developed, and Bradley & Jones 1993 introduced the "Composite Plus Scaling" (CPS) method used by most later large scale reconstructions.^[1][2] Their study was featured in the IPCC Second Assessment Report, and in the United States House Committee on Science its findings were disputed by Pat Michaels.

In 1998 Michael E. Mann, Raymond S. Bradley and Malcolm K. Hughes developed new statistical techniques to produce Mann, Bradley & Hughes 1998 (MBH98), which showed global patterns of annual surface temperature, and included a graph of average hemispheric temperatures back to 1400 with shading emphasising that uncertainties (to two standard error limits) were much greater in earlier centuries.^[3] Jones et al. 1998 independently produced a CPS reconstruction extending back for a thousand years, and Mann, Bradley & Hughes 1999 (MBH99) used the MBH98 methodology to extend their study back to 1000.^[4][5] The term hockey stick was used by the climatologist Jerry Mahlman to describe the pattern this showed, envisaging a graph that is relatively flat to 1900 as forming an ice hockey stick's "shaft", followed by a sharp increase corresponding to the "blade".^[6][7] A version of this graph was featured prominently in the 2001 IPCC Third Assessment Report (TAR), which also drew on Jones et al. 1998 and three other reconstructions to support the conclusion that, in the Northern Hemisphere, the 1990s was likely to have been the warmest decade and 1998 the warmest year during the past 1,000 years.^[5] The graph was featured in publicity, and became a focus of dispute for those opposed to the strengthening scientific consensus that late 20th century warmth was exceptional.^[8]

In 2003, as lobbying over the 1997 Kyoto Protocol intensified, Soon and Baliunas published a paper claiming greater medieval warmth, and on this basis the Bush administration chief of staff Philip Cooney deleted references to climate reconstructions from the first Environmental Protection Agency Report on the Environment. The paper was quickly dismissed by scientists in the Soon and Baliunas controversy, but on July 28, Republican Jim Inhofe spoke in the Senate speech citing Soon and Baliunas to claim "that man-made global warming is the greatest hoax ever perpetrated on the American people".^[9]

Later in 2003, Stephen McIntyre and Ross McKitrick published McIntyre & McKitrick (2003a) disputing the data used in Mann, Bradley & Hughes (1998) paper.^[10] They were given extensive publicity, and met Inhofe as well as making a presentation sponsored by the George C. Marshall Institute and the Competitive Enterprise Institute. In 2004 Hans von Storch published criticism of the statistical techniques as tending to underplay variations in earlier parts of the graph, though this was disputed and he later accepted that the effect was very small.^[11] In 2005 McIntyre and McKitrick published criticisms of the principal components analysis methodology as used in MBH98 and MBH99. Their analysis was subsequently disputed by published papers including Huybers 2005 and Wahl & Ammann 2007 which pointed to errors in the McIntyre and McKitrick methodology. In June 2005 Rep. Joe Barton launched what Sherwood Boehlert, chairman of the House Science Committee, called a "misguided and illegitimate investigation" into the data, methods and personal information of Mann, Bradley and Hughes. At Boehlert's request a panel of scientists convened by the National Research Council was set up, which reported in 2006 supporting Mann's findings with some qualifications, including agreeing that there were some statistical failings but these had little effect on the result.^[12] Barton and U.S. Rep. Ed Whitfield requested Edward Wegman to set up a team of statisticians to investigate, and they supported McIntyre and McKitrick's view that there were statistical failings, although they did not quantify whether there was any significant effect. They also produced an extensive network analysis which has been discredited by expert opinion and found to have issues of plagiarism.

Congressional investigations

The increasing politicisation of the issue was demonstrated when,^[13] on 23 June 2005, Rep. Joe Barton, chairman of the House Committee on Energy and Commerce wrote joint letters with Ed Whitfield, Chairman of the Subcommittee on Oversight and Investigations, referring to issues raised by the 14 February 2005 article in the Wall Street Journal and demanding full records on climate research. The letters were sent to the IPCC Chairman Rajendra Pachauri, National Science Foundation Director Arden Bement, and to the three scientists Mann, Bradley and Hughes.^[14] The letters told the scientist to provide not just data and methods, but also personal information about their finances and careers, information about grants provided to the institutions they had worked for, and the exact computer codes used to generate their results.^[15]

Sherwood Boehlert, chairman of the House Science Committee, told his fellow Republican Joe Barton it was a "misguided and illegitimate investigation" into something that should properly be under the jurisdiction of the Science Committee, and wrote "My primary concern about your investigation is that its purpose seems to be to intimidate scientists rather than to learn from them, and to substitute congressional political review for scientific review." Barton's committee spokesman sent a sarcastic response to this and to Democrat Henry A. Waxman's letter asking Barton to withdraw the letters and saying he had "failed to hold a single hearing on the subject of global warming" during eleven years as chairman, and had "vociferously opposed all legislative efforts in the Committee to address global warming .... These letters do not appear to be a serious attempt to understand the science of global warming. Some might interpret them as a transparent effort to bully and harass climate change experts who have reached conclusions with which you disagree." The U.S. National Academy of Sciences (NAS) president Ralph J. Cicerone wrote to Barton that "A congressional investigation, based on the authority of the House Commerce Committee, is probably not the best way to resolve a scientific issue, and a focus on individual scientists can be intimidating", and proposed that the NAS should appoint an independent panel to investigate. Barton dismissed this offer.^[16][17]

Mann, Bradley and Hughes sent formal letters giving their detailed responses to Barton and Whitfield. On 15 July, Mann wrote emphasising that the full data and necessary methods information was already publicly available in full accordance with National Science Foundation (NSF) requirements, so that other scientists had been able to reproduce their work. NSF policy was that computer codes "are considered the intellectual property of researchers and are not subject to disclosure", as the NSF had advised McIntyre and McKitrick in 2003, but notwithstanding these property rights, the program used to generate the original MBH98 temperature reconstructions had been made available at the Mann et al. public ftp site. ^[18]

Many scientists protested against Barton's investigation, with 20 prominent climatologists questioning his approach.^[19] Alan I. Leshner wrote to him on behalf of the American Association for the Advancement of Science expressing deep concern about the letters, which gave "the impression of a search for some basis on which to discredit these particular scientists and findings, rather than a search for understanding."^[20] He stated that MBH had given out their full data and descriptions of methods, and were not the only evidence in the IPCC TAR that recent temperatures were likely the warmest in 1,000 years; "a variety of independent lines of evidence, summarized in a number of peer-reviewed publications, were cited in support". Thomas Crowley argued that the aim was intimidation of climate researchers in general, and Bradley thought the letters were intended to damage confidence in the IPCC during preparation of its next report.^[21] A Washington Post editorial on 23 July which described the investigation as harassment quoted Bradley as saying it was "intrusive, far-reaching and intimidating", and Alan I. Leshner of the AAAS describing it as unprecedented in the 22 years he had been a government scientist; he thought it could "have a chilling effect on the willingness of people to work in areas that are politically relevant."^[15] Benjamin D. Santer told the New Scientist "There are people who believe that if they bring down Mike Mann, they can bring down the IPCC."^[22]

Congressman Boehlert said the investigation was as "at best foolhardy" with the tone of the letters showing the committee's "inexperience" in relation to science. Barton was given support by global warming denier Myron Ebell of the Competitive Enterprise Institute, who said "We've always wanted to get the science on trial ... we would like to figure out a way to get this into a court of law", and "this could work".^[21] In his Junk Science column on Fox News, Steven Milloy said Barton's inquiry was reasonable.^[23]

In November 2005, Science Committee chair Sherwood Boehlert requested the National Academy of Science to arrange a review of the matter, and its National Research Council set up a special committee to investigate and report.^[24]

National Research Council Report

At the request of the U.S. Congress, initiated by Representative Sherwood Boehlert as chairman of the U.S. House of Representatives Committee on Science, a special "Committee on Surface Temperature Reconstructions for the Past 2,000 Years" was assembled by the National Research Council to quickly prepare a concise report. The NRC Committee, chaired by Gerald North, consisted of 12 scientists and statisticians from different disciplines. Its task was "to summarize current scientific information on the temperature record for the past two millennia, describe the main areas of uncertainty and how significant they are, describe the principal methodologies used and any problems with these approaches, and explain how central is the debate over the paleoclimate temperature record to the state of scientific knowledge on global climate change."^[25] The NRC report went through a rigorous review process involving 15 independent experts.^[26] The report provided a summary and an overview, followed by 11 technical chapters covering the instrumental and proxy records, statistical procedures, paleoclimate models, and the synthesis of large scale temperature reconstructions with an assessment of the "strengths, limitations, and prospects for improvement" in techniques used.^[27]

The NRC committee's report (the North Report) was published on 22 June 2006.^[28] Committee member John Michael Wallace said that "Our conclusion is that this recent period of warming is likely the warmest in the last millennium", and added that "This doesn't change the scientific landscape in terms of the greenhouse warming debate".^[29] In its summary, the NRC committee noted the development of large-scale surface temperature reconstructions, especially MBH98 and MBH99, and highlighted six recent reconstructions: Huang, Pollack & Shen 2000, Mann & Jones 2003, Hegerl et al. 2006, Oerlemans 2005, Moberg et al. 2005 and Esper, Cook & Schweingruber 2002. Its main findings were; 20th century instrumentally measured warming showed in observational evidence, and can be simulated with climate models, large-scale surface temperature reconstructions "yield a generally consistent picture of temperature trends during the preceding millennium", including the Medieval Warm Period and the Little Ice Age, "but the exact timing and duration of warm periods may have varied from region to region, and the magnitude and geographic extent of the warmth are uncertain." It concluded "with a high level of confidence that global mean surface temperature was higher during the last few decades of the 20th century than during any comparable period during the preceding four centuries", justified by consistent evidence from a wide variety of geographically diverse proxies, but "Less confidence can be placed in large-scale surface temperature reconstructions for the period from 900 to 1600", and very little confidence could be assigned to hemispheric or global mean surface temperature estimates before about 900.^[30]

The NRC committee stated that "The basic conclusion of Mann et al. (1998, 1999) was that the late 20th century warmth in the Northern Hemisphere was unprecedented during at least the last 1,000 years. This conclusion has subsequently been supported by an array of evidence that includes both additional large-scale surface temperature reconstructions and pronounced changes in a variety of local proxy indicators". It said "Based on the analyses presented in the original papers by Mann et al. and this newer supporting evidence, the committee finds it plausible that the Northern Hemisphere was warmer during the last few decades of the 20th century than during any comparable period over the preceding millennium", though there were substantial uncertainties before about 1600. It added that "Even less confidence can be placed in the original conclusions by Mann et al. (1999) that 'the 1990s are likely the warmest decade, and 1998 the warmest year, in at least a millennium' because the uncertainties inherent in temperature reconstructions for individual years and decades are larger than those for longer time periods and because not all of the available proxies record temperature information on such short timescales." It noted that "Surface temperature reconstructions for periods prior to the industrial era are only one of multiple lines of evidence supporting the conclusion that climatic warming is occurring in response to human activities, and they are not the primary evidence."^[31]

At the press conference, North said of the MBH papers that "we do roughly agree with the substance of their findings. There is a small disagreement over exactly how sure we are."^[32] All three from the NRC committee panel said it was probable, though not certain, that current warming exceeded any previous peak in the last thousand years.^[24] When asked if they could quantify "less confidence" and "plausible", Bloomfield explained that their wording reflected the panel's scientific judgements rather than well defined statistical procedures, and "When we speak of 'less confidence' we're more into a level of sort of 2 to 1 odds, which IPCC, they interpreted 'likely' as that level, roughly 2 to 1 odds or better."^[29][32]

Various criticisms of the MBH statistical methods were discussed in Chapter 11 in the context of more recent research which explored ways of addressing these problems, and showed greater amplitude of temperature variations over 1000 to 2000 years. Recent papers cited included Wahl & Ammann 2007. On McIntyre and McKitrick's criticism of principal component analysis as tending to bias the shape of the reconstructions, it found that "In practice, this method, though not recommended, does not appear to unduly influence reconstructions of hemispheric mean temperature", and reconstructions using other methods were qualitatively similar. Some of the criticisms of validation techniques were more valid than others, these issues and the effect on robustness of the choice of proxies contributed to the committee's view of increased uncertainties. They called for further research into methods and a search for more proxies for earlier periods.^[33]

At the press conference the three NRC panellists said they found no evidence supporting the allegations of inappropriate behaviour such as data manipulation, or "anything other than an honest attempt to construct a data analysis procedure". Bloomfield as a statistician considered all the choices of data processing and methods to have been "quite reasonable" in a "first of its kind study". He said "I would not have been embarrassed by that work at the time if I'd been involved in it". In response to a question from Edward Wegman on the MBH use of principal components analysis, Bloomfield said this had been reviewed by the committee along with other statistical issues, and "while the issues are real, they had a very minimal effect, not a material effect on the final reconstruction."^[32]

Committee on Energy and Commerce Report (Wegman Report)

Barton dismissed the offer of a joint investigation with an independent panel appointed by the U.S. National Academy of Sciences (NAS),^[16] and on 1 September 2005 statistician Edward Wegman was contacted about giving testimony. Bardon's staffer then met Wegman, and explained that the United States House Energy Subcommittee on Oversight and Investigations wanted expert opinion on the validity of criticisms of the Mann, Bradley & Hughes 1999 reconstruction. Wegman set up a team of statisticians, and over the next nine months Barton's staffer provided them with material to review.^[34] The team included Wegman, Yasmin H. Said who had been his graduate student, and statistician David W. Scott, all statisticians with no expertise in climatology or other physical sciences.^[35] They were assisted by two others;^[36] Wegman testified that one of his graduate students did the code for them,^[37] and another of his graduate students provided a section of the report in draft, but was not an author of the report.^[38]

Lacking Barton's agreement, Boehlert's Science Committee independently requested the NAS in November to commission the National Research Council Report. On 10 February 2006 the Wall Street Journal revealed Barton's contact with Wegman when it reported that "people familiar with the matter" said that Barton had already requested an analysis of the hockey stick from statistician Edward Wegman. It said that the deputy staff director for Barton's committee had made a statement that closer study was needed because of anticipated costs of mitigating climate change, and that Barton's concerns were "unlikely" to be fully addressed by the NAS.^[39]

In an editorial dated 14 July 2006, the Wall Street Journal announced that a report commissioned by the Energy and Commerce Committee was due to be released that day. It gave a preview of the conclusions of the report, which had been prepared by three statisticians.^[40] The committee chairman U.S. Rep. Joe Barton issued a press release giving a summary of the report's findings, with quotations from the report.^[36][41]

The Wegman Report was not peer reviewed in the same way as the NRC Report, but was sent out to a number of referees before it was released. Wegman lacked the procedures and staff which the NRC had in place to organise peer review.^[42] One of the referees, Grace Wahba, later said she was given the report only 3 days in advance, and her criticisms were ignored.^[43]

Discussion and hearings

The Wegman Report was discussed at hearings of the United States House Energy Subcommittee on Oversight and Investigations under its chairman U.S. Rep. Ed Whitfield. The first hearing was held on 19 July 2006, five days after the report was announced and released.^[40][44] Michael E. Mann was given only three days notice of the hearing and advised the committee that he was unable to attend on that date due to a commitment to look after his baby daughter, but the hearing went ahead with testimony from paleoclimatologist Tom Crowley who Mann recommended in his absence. The other scientists giving testimony were Gerald North, chairman of the National Research Council panel that had produced the NRC report, Tom Karl, director of the National Climatic Data Center, and Hans von Storch. Wegman and Steve McIntyre also testified at the hearing.^[45][46]

A second hearing was arranged on 27 July 2006, and heard testimony from Mann, John Christy, a climate scientist at the University of Alabama in Huntsville, Ralph J. Cicerone, president of the National Academy of Sciences, and Jay Gulledge of the Pew Center on Global Climate Change. The hearing also heard testimony from Wegman and McIntyre.^[46]

In an interview, North stated that the politicians at the hearing at which the Wegman Report was presented "were twisting the scientific information for their own propaganda purposes. The hearing was not an information gathering operation, but rather a spin machine."^[42]

• The American Statistical Association held a session on the role of statistics in public policy debates about climate change at the September 2006 Joint Statistical Meetings. Speakers were Edward Wegman, John Michael Wallace, and Richard L. Smith.^[47]

Issues raised by the Wegman Report

The report opened with a statement that its remit was to provide "an independent verification of the critiques" of MBH98 and MBH99 by McIntyre and McKitrick (MM), "as well as the related implications". They produced a social network analysis to imply problems of peer review not being independent, suggested that climatologists worked in isolation from statisticians, and alleged that there was inadequate sharing of methods and data.^[48]

Critiques of MBH98 and MBH99 by McIntyre and McKitrick

• The report stated "We have sought to reproduce the results of MM in order to determine whether their criticisms are valid and have merit", and concluded "In general, we found MBH98 and MBH99 to be somewhat obscure and incomplete and the criticisms of MM03/05a/05b to be valid and compelling".^[49]
  • Mann's immediate response was that the report "uncritically parrots claims by two Canadians (an economist and an oil industry consultant) that have already been refuted by several papers in the peer-reviewed literature inexplicably neglected by Barton's 'panel'. These claims were specifically dismissed by the National Academy in their report just weeks ago."^[50]
  • In the testimony of Jay Gulledge of the Pew Center on Global Climate Change, the report had reproduced MM's work verbatim but had only made a partial assessment of its validity, and had failed to assess the merits of their criticisms. The report made no attempt to find if MM's suggested corrections made any significant difference to the outcome. These suggested corrections had been thoroughly tested by Wahl & Ammann 2006, who had found only two of them to be valid and these had very little effect on the outcome.^[51]
  • Gerald North gave testimony that "Dr. Wegman's criticisms of the statistical methodology in the papers by Mann et al. were consistent with our findings", referring to the NRC report which had found that the methodology did not have an undue effect on the graphs. In his view, "none of the statistical criticisms that have been raised by various authors unduly influence the shape of the final reconstruction. This is attested to by the fact that reconstructions performed without using principal components yield similar results."^[52]
  • RealClimate stated that the result of fixing the alleged errors made no practical difference to the overall reconstruction, and the hockey stick shape remained. Similarly, studies that use completely different methodologies also yield very similar reconstructions.^[53]
• The Wegman Report stated that the MBH method creates a hockey-stick shape even when supplied with random red noise (Figure 4.4). "It is not clear that Dr. Mann and his associates even realized that their methodology was faulty at the time of writing the MBH paper." (Section 4)
  • On 23 July David Ritson, Emeritus Professor of Physics at Stanford University, emailed Wegman to ask if the Wegman team had used the same red noise method as MM, and if they had carried out a basic procedure to check if the Principal Component (PC1) they showed in their illustrations was due to a systemic signal or to random noise. Wegman did not reply to this, or to follow-up emails which pointed out that MM had provided Ritson with their code on 6 November 2004, and this code had an error which produced noise with "extraordinarily high persistence" resulting in the hockey stick shapes. The Wegman Report did not discuss this improper procedure or provide specification data for their own results. After Ritson copied this correspondence to Mann and North, he wrote to congressman Henry Waxman on 16 August 2006 about this key information missing from the Wegman Report.^[54] In his written evidence to questions raised at the hearing, Mann said that "the errors that Dr. Ritson has identified in Dr. Wegman’s calculations appear so basic that they would almost certainly have been detected in a standard peer review."^[55]
  • Hans von Storch testified that the tendency to produce hockey-stick shapes from random data would only apply if there were no other significant signals, and "in very many practical situations it would not show up."^[56]
• The report stated "The controversy of Mann’s methods lies in that the proxies are centered on the mean of the period 1902-1995, rather than on the whole time period", and the net effect of this decentering was to produce a "hockey stick" shape.
  • The MBH reconstruction involved temperature records of various lengths, the shortest being the instrumental record from 1902 to 1980, and their convention centered data over this modern calibration period.^[57] Their procedure used principal component analysis (PCA) to find the leading patterns of variation (PC1, PC2, PC3 etc.), and used an objective selection rule procedure to establish how many significant principal components had to be kept to accurately represent the full dataset. The Wegman Report only dealt with PC1, and Mann describes it as "trivially true, but in reality totally inconsequential" that changing the centering period will change the order of principal components so that PC1 shows a different pattern. MM had confined their calculation to PC1 and PC2; like Wegman they ignored the selection rule and eliminated the significant "hockey stick" pattern which was in the original data.^[58]
  • In testimony, von Storch stated his working group (von Storch & Zorita 2005) had examined how seriously the "biased centering" reported by Wegman would affect overall results, and "the effect is very minor. It does not mean that it is not a glitch but it really doesn’t matter here, at least to the extent we could test it."^[59]
• The report said that MBH method creates a PC1 statistic dominated by bristlecone and foxtail pine tree ring series (closely related species). However, there is evidence in the literature that the use of the bristlecone pine series as a temperature proxy may not be valid (suppressing "warm period" in the hockey stick handle); and that bristlecones do exhibit CO₂-fertilized growth over the last 150 years (enhancing warming in the hockey stick blade).
  • Barton's letter had asked about these proxies, and in reply Mann referred to the detailed examination in MBH99 of the effect that removing these proxies had on the validity of the reconstruction.^[60]
  • The NRC report, as stated in North's testimony, found that modern CO₂ fertilisation affected these trees at some but not all of the places where samples had been taken for the Mann study, and in view of calibration difficulties it was easier not to use them. However, "strip-bark data are considered suspect only after the modern increase in atmospheric carbon dioxide concentrations. This is why other studies that rely on strip-bark pine records only use them to infer past temperatures prior to 1850.^[61]
  • Crowley described this as a "red herring", and testified that his own studies using different methodology had results which were remarkably close to MBH, "bristlecone pine or no bristlecone pine". He said "If the bristlecone pine record is removed from the composite of a dozen or so records, it will show slightly greater warming in the Middle Ages. But one record can only make so much a difference when it is averaged among a dozen, especially since the general shape of the bristlecone pine record is comparable to the other records."^[62]
  • In his prepared statement to the hearing, Gulledge noted the Wahl & Ammann 2006 study found that excluding the bristlecone and foxtail pine proxies showed slightly more warming in the early 15th century, but had less effect on earlier and later periods. Overall, this made very little difference and did not undermine the main conclusion that medieval warmth had not reached late 20th century temperatures.^[63]
• Many of the same proxies are reused in most of the "independent studies" so these "cannot really claim to be independent verifications."^[41]
  • Tom Karl, director of the National Climatic Data Center, testified that the TAR had reviewed multiple proxies which were independent of those used by MBH, and supported the conclusion that current warming exceeded that in earlier centuries.^[64]
  • Mann's response to Barton's questions, as submitted to the hearing, noted that more proxy data had been published since 1999, and had been used with new methods in reconstructions which confirmed that late 20th century warming was exceptional, including Moberg et al. 2005 which covered 2,000 years.^[60]

• As Barton highlighted in his press release, the IPCC Third Assessment Report had assessed the 1990s as the hottest decade and 1998 the hottest year in 1,000 years.^[36] The Wegman report concluded that assessment could not be supported by the MBH studies, and said that "The cycle of Medieval Warm Period and Little Ice Age that was widely recognized in 1990 has disappeared from the MBH98/99 analyses, thus making possible the hottest decade/hottest year claim." On the basis of their Figure 4.5 showing their "digitised" version of a schematic from the IPCC First Assessment Report (FAR), they said "It is clear that at least in 1990, the Medieval Warm Period was thought to have temperatures considerably warmer than the present era.Wegman, Said & Scott 2006, pp. 34, 49
  • Jay Gulledge stated that the scientific understanding of the Medieval Warm Period had developed over time, and the schematic diagram as used by Wegman, McIntyre and the Wall Street Journal "is not a plot of data and is inappropriate as a comparison to MBH". The 1990 IPCC FAR had been clear that even then there was evidence showing regional rather than global medieval warming. The MBH analysis had emphasised uncertainties, and its overall robustness had been reaffirmed by Wahl & Ammann 2006 and by the NRC report.^[65]
  • When questioned about the WR Figure 4.5, Wegman said he "had not been able to obtain a copy of the 1990 report", but believed that the figure "was related to the European temperatures and was a cartoon– essentially a cartoon" which they were simply using as an example. When asked about the data supporting the figure, he said "I take no responsibility for what IPCC did in 1990. There is no way I could do that. Their data is not available to me. In fact, the reason it was digitized was that I had to go back and construct it from their picture. That doesn’t mean no data exist."^[66]

Peer review: social network analysis

• In the WR, "we judge that there was too much reliance on peer review, which was not necessarily independent." It continued, a "social network of authorships in temperature reconstruction is described of at least 43 authors with direct ties to Mann by virtue of having coauthored papers with him. The findings from this analysis suggest that authors in the area of paleoclimate studies are closely connected and thus ‘independent studies’ may not be as independent as they might appear on the surface.
  • At the hearings, Wegman stated this was a "hypothesis", and "should be taken with a grain of salt." He listed 6 people who had participated in his own informal peer review process via email after the report was finalized and said they had no objection to the subcommittee submitting it. He defined the social network as peer reviewers that had "actively collaborated with him in writing research papers" and answered that none of his peer reviewers had.^[67]
  • One of those listed by Wegman was the statistician Grace Wahba, who told North by email "Hey they used my name and they said I was a referee. He sent it to me about 3 days beforehand and I sent him a bunch of criticisms which they didn‘t take into account."^[43]
  • At the hearings, North criticised aspects of this analysis, which had not been available for the NRC to examine. He "was not impressed by the social network analysis" and differed from "the report's conclusions on this subject". Indeed, he would look favourably on an application for tenure by a young scientist who "got busy and found himself 43 coauthors."^[68]
  • In John Quiggin's opinion, the social network analysis was not based on meaningful criteria, did not prove a conflict of interest and did not apply at the time of the 1998 and 1999 publications. Such a network of co-authorship is not unusual in narrowly defined areas of science.^[69]
  • When allegations of plagiarism in this section were examined, Wegman said that material in it had been "basically copied and pasted" by a student who was the "most knowledgeable" person about such analyses on his team, as she had taken a one-week course on network analysis with Kathleen Carley of Carnegie Mellon University. Carley described the paper based on this section as "more of an opinion piece", lacking the data needed to support its argument.^[38][70]
  • The paper based on this social network analysis, published by Wegman and his former student Said, reached the conclusion that eminent scientists ought not work together, and that the findings of studies would be less biased when a "principal author tends to co-author papers with younger colleagues who were his students".^[70]

Interactions between climatologists and statisticians

• The Wegman Report said that there was no evidence that Mann or any of the other authors in paleoclimatology studies have had significant interactions with mainstream statisticians. It alleged that the paleoclimate community was relatively isolated, relying heavily on statistical methods but not seeming to interact with "the statistical community".
  • In his written response to questions, Mann said Wegman appeared to have written this without making any investigation, and many statisticians working in climatology had been offended by Wegman's claim. The NRC committee included the statisticians Douglas Nychka and Peter Bloomfield who had both worked with climatologists, and Mann himself had been a member from 2003 to 2005 of the American Meteorological Society Committee on Probability and Statistics along with other scientists and statisticians. The National Center for Atmospheric Research Geophysical Statistics Project had provided such interaction for over a decade, and more than 24 of those participating had subsequently gained doctorates in statistics. Mann as a graduate student had participated in its inaugural workshop in 1994. Its head Doug Nychka had been directly consulted for Wahl & Ammann 2006. Textbooks had been published on statistical climatology, and von Storch testified that he had co-authored "Statistical Analysis in Climate Research" with the statistician Francis Zwiers.^[71]

Sharing of data and methods

• The Wegman Report said "Sharing of research materials, data, and results is haphazard and often grudgingly done. We were especially struck by Dr. Mann’s insistence that the code he developed was his intellectual property and that he could legally hold it personally without disclosing it to peers. When code and data are not shared and methodology is not fully disclosed, peers do not have the ability to replicate the work and thus independent verification is impossible."
  • Mann indicated in testimony that the methods and data had been available since May 2000, including the necessary algorithms, in full accordance with National Science Foundation requirements, but NSF policy was that computer codes were proprietary and not subject to disclosure. Despite this, the full code used for MBH98 had been made public.
  • In his written testimony, John Christy said that when he and Roy Spencer had been asked by Frank Wentz et al., "we provided sections of our code and relevant data files. By sharing this information, we opened ourselves up to exposure or a possible problem which we had somehow missed, and frankly this was not personally easy. On the other hand, if there was a mistake we wanted it fixed." Rep. Henry Waxman contacted Wentz, and read out his response that "Dr. Christy has never been willing to share his computer code in a substantial way", but had replied to their request by writing "I don't see how sharing code would be helpful". In Wentz's view, "he simply didn't want us looking over his shoulder, possibly discovering errors in his work. So we had to take a more tedious trial-and-error approach to uncovering the errors in his methods." Christy told Waxman that "We shared with them the parts of the code that they were most concerned about."
  • When Ritson repeatedly emailed Wegman asking for clarification of the code and data used by the Wegman team to produce random red noise, Wegman failed to answer.^[54] After congressman Waxman asked him about this, Wegman replied on 1 September 2006 that his spam filter was set to stop all emails related to the topic, and his team did not look at their emails often. He added "It is not clear to me that before the journal peer review process is complete that we have an academic obligation to disclose the details of our methods." On 15 September Waxman wrote formally to Wegman asking for this information, and commenting that the Wegman report had said that methodology had to be fully disclosed to allow independent verification. In his written response to questions raised at the hearing, Mann said "It would appear that Dr. Wegman has completely failed to live up to the very standards he has publicly demanded of others." Ritson commented in September 2007 that Wegman had still not responded to further requests, and while he could sympathise with someone with a weak position not wanting to answer questions, "in this instance Wegman has made a central point of the need for openness in science, and never, over my academic career, has anyone avoided actions by promising to implement a series of steps which they apparently had no intention of ever doing."^[55][72]

Plagiarism charges against Wegman

Wegman's institution, George Mason University, confirmed in October 2010 that they were investigating misconduct charges, following a March 2010 formal complaint by Raymond S. Bradley alleging plagiarism and fabrications in the Wegman Report. A 250-page study by computer scientist John Mashey, posted on the "Deep Climate" website, claims that 35 of the 91 pages in the Wegman Report were plagiarized, and "often injected with errors, bias and changes of meaning." Wegman responded that he was "very well aware of the report", but at the university's request would not comment further until all issues were settled.^[73] Reviews by outside experts contacted by USA Today found plagiarism from textbooks which was obvious and inappropriate; the social network analysis section had also been partly copied from Wikipedia. Wegman said there was "speculation and conspiracy theory" in John Mashey's analysis, and said that "[t]hese attacks are unprecedented in my 42 years as an academic and scholar." He stated that the Wegman Report never "intended to take intellectual credit for any aspect of paleoclimate reconstruction science or for any original research aspect of social network analysis."^[74]

Social network analysis paper

As an extension of the part of the Wegman Report which used social network analysis to suggest that there had been inappropriate close collaboration between some climate scientists, Wegman and Said published a paper (since retracted) in 2008 in the journal Computational Statistics & Data Analysis where they suggested "that certain styles of co-authorship lead to the possibility of group-think, reduced creativity, and the possibility of less rigorous reviewing processes". They concluded they had provided "insight into the why certain fields of study may have migrated into a more politically driven framework."^[75] After computer scientist Ted Kirkpatrick of the Simon Fraser University read the "Deep Climate" website allegations of plagiarism, he made a formal complaint to the journal.^[70] On 16 March 2011, Wegman sent an email to the journal saying that a student "had basically copied and pasted" work by other authors into the Wegman Report, and this text had been used in the journal paper without acknowledgement. He said that "We would never knowingly publish plagiarized material". In May 2011 the journal's editor, Stanley Azen of the University of Southern California, announced that the journal was retracting the paper, because it used portions of other authors' writings without sufficient attribution.^[76][77] John Dahlberg of the United States Office of Research Integrity indicated that plagiarism could result in sanctions. A George Mason University spokesman declined to comment and said it was a "personnel matter".^[70]

The manuscript of the paper had been submitted on 8 July 2007 and accepted for publication on 14 July 2007. Network analysis expert Kathleen Carley described it as an opinion piece which speculated that collaboration between scientists "leads to peer review abuse. No data is provided to support this argument". Wegman's student had attended a one-week course taught by Carley, thus becoming what Wegman described as the "most knowledgeable" of his contributors on the topic. Following the GMU inquiry, the student issued a statement that she had been "Dr. Wegman's graduate student when I provided him with the overview of social network analysis, at his request. My draft overview was later incorporated by Dr. Wegman and his coauthors into the 2006 report. I was not an author of the report." She had met with a GMU misconduct committee, and said that "My academic integrity is not being questioned."^[38]

Disciplinary procedure

A Nature editorial commented on the implication that the plagiarised material in the retracted paper was likely to also be present in the earlier "infamous" Wegman Report, including allegations against Mann and his co-authors which had frequently been cited by climate-change deniers. The George Mason University's policies indicated that its initial inquiry should have been completed within 12 weeks of the original complaint, and although 14 months had passed without this being resolved, there were loopholes for extensions. It said that the university should "take the initiative to move investigations along as speedily as possible while allowing time for due process. Once an investigation is complete, the institution should be as transparent as it can about what happened", especially where public funds were involved.^[78]

George Mason University provost Peter Stearns announced on 22 February 2012 that charges of scientific misconduct had been investigated by two separate faculty committees, and that the one investigating the 2006 Wegman Report gave a unanimous finding that "no misconduct was involved". Stearns stated that "Extensive paraphrasing of another work did occur, in a background section, but the work was repeatedly referenced and the committee found that the paraphrasing did not constitute misconduct". He said that the 2008 social network analysis paper was investigated by a separate committee which unanimously found "that plagiarism occurred in contextual sections of the (CSDA) article, as a result of poor judgment for which Professor Wegman, as team leader, must bear responsibility", and Wegman was to receive an "official letter of reprimand". The investigation reports were to be sent on to federal authorities, but would not be made public. Bradley described the split result as "an absurd decision" which would encourage GMU students to think it acceptable to copy work without attribution.^[79]

Notes

References in chronological sequence

External links

• Ad Hoc Committee Report on the 'Hockey Stick' Global Climate Reconstruction

This page was last edited on 17 March 2024, at 14:19