Bishop Hill Climategate and HADCRUT
May 28, 2012 Climate: Allen Climate: CRU Climate: Surface I'm sure most readers are aware of Steve McIntyre's post on the Myles Allen video. The comments thread is also very interesting. I was particularly taken with Lucia Liljegren's response to a comment made by Myles on the subject of the relevance of the temperature records to CRUTEM. This is what Myles said:
I stand by the assertion that, thanks to the sloppy coverage the affair received in the media, it wasn’t just Sarah Palin who got the impression that the instrumental temperature record was seriously compromised: The Times opened the relevant story with “A science blogger has uncovered a catalogue of errors in Met Office records…”
Lucia's response should be read in full, but this excerpt gives a flavour
I would suggest that the main reason for this “sloppy coverage” was that reporters turned to people trying to rebut those discussing climategate at blogs and in forums. Some people people who (like you) might prefer to discuss the thermometer record rather than misbehavior of scientists or what “hide the decline” meant, diverted the discussion to the thermometer record.
I strongly suspect the behavior of the scientists who wanted to suppress discussion of climategate succeeded in giving the media the incorrect impression that climategate was about the thermometer record is one of the reasons much of the media, some politicians, and Sarah Palin developed the impression climategate is about the thermometer record. That you can show they were confused about what people at blogs and forums were posting about merely shows you don’t know what it was about.
The thing that struck me was that the errors found in HADCRUT had almost nothing to do with Climategate anyway. When the initial furore over the East Anglia emails broke out, the Met Office for some reason decided to release its data and code. This was subsequently analysed by John Graham-Cumming, who found some minor errors in it. The related Times article that Myles refers to can be seen here in the Internet Archive. Readers will note that it contains no mention of the University of East Anglia or the Climategate emails and in fact dates to several months after the UEA disclosures. The two stories are linked only by the fact that the Met Office's decision to release its data and code seems to have been prompted by the lack of trust in climatologists engendered by the UEA disclosures.
Nevertheless, it's interesting to ponder the way the Climategate story came to be so much intertwined with the surface temperature records - Sarah Palin's initial comment on the day the story hit the press, Beddington's diversion of the Russell panel's work, the SciTech committee's decision to look at CRUTEM and the work of John Graham-Cumming. Cause and effect are hard to determine here.
Reader Comments (69)
The Guardian accounted for the vast majority of comment on Climategate in the British media, with 12 articles in the first week alone. (Do you want me to put up references?)
In the second, by Bob Ward himself, he talks of: Despite contributing several articles himself, 6 months later, Monbiot opened the Guardian debate by asking Steve McIntyre about the HADCRUT data.In the first Guardian article by Hickman and Randersen, Bob Ward is quoted as saying
Myles is a frequent contributor to Guardian Environment, so has been well placed to correct any misunderstanding.
Myles, you are still spinning! We don't agree, I stand by my statement on this page from 6 days ago that seems to have sparked all this off, that your talk was misleading and a misrepresentation of climategate. If you're still not sure what the objection is there's probably no point going over it again at this stage, you can read Andrew's new book!
What I meant was that I don't think that the crime of your misleading statements merited the punishment of almost a week's worth of sometimes abusive blog comments, ridicule, cartoons and caption competitions. I'm now with Jack Savage who said "can we move on".
I am hoping and expecting that Andrew's new book will provide the best overview yet on the substance and significance of issues that became known as "Climategate"....
Clearly the 'gate' suffix to echo a famous US political scandal indicated that (at least for people who use this term "Climategate" as I do) the whole set of issues and behaviors is a scandal. To dismiss it as only about one tiny correction in HADCRUT in the 1870s seems to imply there was nothing else "there" at all.
For the moment, I recommend to all Steve McIntyre's latest comment here.
I do thank Myles Allen for continuing to engage on these matters. If I may make a mischievous suggestion (and it's merely a timid suggestion, I know he must be a very busy man), he could impress all of us if he were to look seriously at Andrew's new book (I think it is to be released soon) and to consider offering his critical response (here or elsewhere) after reviewing the whole range of what Andrew and others have considered important under the rubric of "Climategate"....
Meanwhile, some of us may continue to be mired in musings about "irrelevancies" until we are set straight.
Dear Paul and Rhoda,
How frustrating -- I had no idea anyone had posted on the Wolfson lecture. I wish someone had told me. And we have just wasted days arguing over climategate. Yet another example of how this issue continues to divert energy from the really interesting questions.
I did try and divert the conversation back to negative liberty, but without success. I accept that the Communicate2011 talk wasn't a comprehensive survey of climategate. It wasn't meant to be: I wouldn't pretend to be able to survey the whole saga in a 10-minute talk. I was using the example of the confusion engendered by people thinking climategate affected the surface temperature record, which we can lay squarely at the door of media coverage of the issue at the time, as one of the factors behind the current confusion. I don't think, in all this exchange, anyone has actually disputed the core statement that the 0.02K correction was the only revision to a published dataset used in detection and attribution studies to have emerged from the entire affair.
As Mike Hulme points out, climategate means many things to many people, but I think we do all agree that it was not about the surface temperature record: the problem I was highlighting was that lots of people thought it was about the surface temperature record. As James Delingpole put it, "This matters because ... HadCRUT". I have made that remark before, for example to a meeting of the UK Association of Science Journalists, and no-one disputed it -- and these are the people responsible for the misleading coverage that I was criticising. If it had been made completely clear all along that it was only about scientific process, the presentation of results and a long-standing dispute among dendroclimatologists, the public would have been better able to keep the whole affair in perspective.
Just to be clear, Rhoda, I never said anyone was "bent" -- I am not in the business of making allegations of dishonesty. I said there were problems, on all sides. I don't think that is a contentious statement.
I don't think its OK to pour out all this bile and then just say "let's move on". I never asked for all this -- but Andrew has now started 4 separate threads on the issue. I don't think I have much to add to my long post to you both earlier this afternoon. Perhaps if Andrew could elevate that post and this one up to the top, to address his core allegation that I don't understand climategate, that might be a good way of drawing a line under all this.
Myles
Dear Myles,
I too have been feeling some sympathy for you over the last few days. As academics (I'm a professor of chemistry in Bristol), we are not used to being scrutinized by outsiders in an aggressive way, and some rather pointed things have been written to you and about you. For example, the thread about Will.I.am - though intended, I am sure, more in the vein of gentle lampooning of a rather pompous rapper, must have felt like a rather personal attack. I saw your comment saying that Intel had foisted him on you. I can't imagine quite what a pain I would find it to be were that to happen to me. That being said, you work in a very high profile area, and this has benefits - readily written impact statements, a generous flow of grant funding to the field over the last 10 or 20 years - but it also has some responsibilities. If you look at it that way, you can see the many threads here and elsewhere that have revolved around your name as being, in a strange kind of way, rather flattering: a recognition of the importance of your research.
I think you could do a lot of good to the public image of climate science by engaging with some of the more thoughtful comments that have made their way towards you in the last few days. At present, I see you following the pattern that many climate scientists seem to follow, which is to refuse to concede that any of the criticisms have any merit whatsoever.
Two examples: Steve McIntyre wrote a very eloquent and well-reasoned comment on a number of topics, but especially about why he thought your talk at the Communicate session had provided an imbalanced view of climategate. To many people, his argument will look very convincing and worthy of a thoughtful response. Your reply on that thread looks reflexively defensive to me, and I am sure it will to others too. Many people end up giving out mixed messages in a talk, and my experience is that it is especially easy to do so in talks aimed at people outside one's field, as that talk was. Surely it would not be impossible to accept that in retrospect some of Steve's points were good?
Another example: Climategate was largely about proxy studies such as the Hockey Stick. Several people have pointed out to you that these do have some importance (basically they are one of the "many independent lines of evidence" often cited in support of AGW). Yet you seem to see nothing wrong in what scientists have done in this field. In your comment on this thread at 2:32 pm, you write, about the "Hide the Decline" issue:
To many people on the sceptic side, this just does not cut the mustard, and many people will find it intellectually dishonest. It is a standard defence that gets brought up for 'hiding the decline', but anybody who takes time to think about it in an independent way - I assume you have not - can see that it is a poor defence. The claim that tree-rings are robust temperature proxies is not firmly established. So the decline, far from signalling "what is generally agreed to be a spurious cooling", is a serious issue that suggests that proxies may provide very flawed information about temperatures in, say, the Middle Ages.
Best wishes
Jeremy
Dear Myles, you never said anyone was bent. I did. There are problems in so-called scientists doing, well, all the things the CRU team did. Not on all sides. Their actions, obfuscation and bad faith. I wish you could condemn it as I would wish to see it condemned. If you can't, you can't. But that leaves some of us thinking that the house has not been cleaned out and that therefore we cannot trust anything which comes from anyone linked with the IPCC who turns a blind eye to it. I don't think you (the collective you, that is advocates of CAGW sience) can regain lost trust. There is a dynamic of 'won't get fooled again' amongst those whose trust you have lost. It is a mistake to see this as a communication problem.
Myles Allen
But climategate DID affect the public’s confidence in the temperature record, not because it revealed a conspiracy to cook the graphs, but because it showed the scientist in charge threatening to destroy data and rejoicing at the death of a sceptic. There is no "current confusion".We can’t judge the science, (as you and your colleagues endlessly remind us) so we judge the quality of the scientists who do the science. What is a normal person to think of Phil Jones and of his reinstatement after the enquiries? What do you think of him?
Dear Jeremy,
I did accept that the talk did not give a comprehensive overview of climategate. I am not a sociologist of science, nor am I a dendroclimatologist, so I'm not qualified to do so. I was using the climategate story to illustrate how sloppy journalism allowed the public to get entirely the wrong end of the stick. As I keep repeating, no published dataset used in detection and attribution has been significantly revised or withdrawn as a result of the "revelations" in the e-mails, and that is not the impression the public have received. I think that is an important failure of science communication -- which in all this discussion, no one seems to be specifically rebutting. Rather I am hearing a lot of shouting "that's not the point" -- to which I would reply, well, it was my point, and it was a valid point. I am not claiming the talk was perfectly balanced and honed by an expensive legal team: I was doing it as a favour, wrote it on the train down and had no idea it was going to end up on YouTube.
It does hurt when a fellow academic suggests that what I am saying sounds intellectually dishonest. I was just saying I can understand the argument for not showing the data that Keith Briffa had concluded was contaminated, just as I can understand the argument for showing it. When Steve McIntyre or Richard Muller talk about this, they make it all seem completely black-and-white, but it isn't. It all comes down to the dendroclimatologists' confidence that whatever it was that was contaminating the most recent decades would not have contaminated the earlier data. They clearly were sufficiently confident about this to feel comfortable with displaying the data in the way that they did. I'm not a dendroclimatologist, so I don't feel qualified to pronounce either way. But I don't use tree-ring data: perhaps that speaks for itself.
I must say it is very odd to be cast in this as a paid-up supporter of Mann et al. The only role I played in this affair was to insist, as a (not very anonymous) peer reviewer, that they responded properly to the perfectly sensible criticisms raised by von Storch and Zorita, much to Mann et al's irritation. Someone told me this is all discussed in the e-mails. But I guess that wouldn't fit the current narrative. As Roger Pielke put it, everyone has to be on one team or the other.
Myles
When someone is whining about 'an unflattering image on youtube' in the middle of a debate about science and policy - well you can draw your own conclusions. Even that wouldn't matter much, except that the personal vanity which drove such a comment is very likely to be at least partially transferable to the closely held beliefs of that person.
Sport, your bum looks huge when you are wearing the 'layered look' that seems to be the preferred style of those who find the 'nudity' of real science unflattering.
Dear Myles, the problem I have with your talk is that I reject your premise. Useless then to debate the detail of your conclusions. The Science is not settled, and while the bent folks remain in office and the IPPR' warm words' tactics* remain in use, there is a trust problem. It is gone. To me, that is the problem you have, that is the crux, the uniting theme, of the way you have been treated here and elsewhere. You cannot 'move on' until you have dealt with it. Every time you prevaricate you make it worse.
* http://www.ippr.org/images/media/files/publication/2011/05/warm_words_1529.pdf
Myles, thanks for your reply. You're right: the debate is so polarized that everything gets interpreted in such a way that people are assigned to one team or the other. That is sort of what I was saying: when you write about Hide the Decline being a case of "not showing what is generally agreed to be a spurious cooling", many people will immediately classify you as belonging to one of the teams.
I do think that Myles Allen should be commended and appreciated for continuing a discussion that I for one have found helpful. While we all wish to convince him that he should think or do this or that in accordance with our own judgments, he has hung in here through some rough weather and continued the exchange. Also, I want to point out for any who may not have seen it that he has dared to criticize the illustrious Al Gore here.
While from the standpoints of various "skeptics" there is so much more we would like to say about Gore's hype and exaggerations, I am unaware of other examples of climate scientists taking on Gore at all, on anything (I posted something about this on "Unthreaded" for anyone interested). For his trouble Myles earned a verbal onslaught from political commissar Joe Romm at Climate Progress. That shows some "politically incorrect" potential which makes the commissars nervous....
That article you link to, Skiphil, is by Myles himself and he says this as perhaps his most severe criticism of Gore:
'This illustrates an important point: human influence on climate is making some events more likely, and some less likely, and it is a challenging scientific question to work out which are which. Randy Dole and co-workers found no evidence for human influence increasing the risk of the 2010 Russian heatwave, the jury is still out on the Pakistani floods, and has broken up in disarray over hurricane Katrina. So when Gore says: "the environment in which all storms are formed has changed," he isn't actually lying, but he is begging to be misunderstood..'
Not exactly a heavy hit! But he dared to criticise the (g)Oracle, and if he thereby earned the opprobrium of an unhinged activist such as Romm, these two things are going to be on the credit side of his ledger.
Note added within my 14mins: I am also not clear as to the difference of opinion he has with Gore. Gore argues that the 'environment has changed', and therefore it would seem to follow that associated probabilities will have changed. Myles seems to be arguing (video clip, http://www.bishop-hill.net/blog/2012/5/23/myles-allen-on-climategate.html) that the probabilities have changed.
John Shade
Agreed, it was not exactly a severe take-down of Gore. My point is merely that it is nice to see a prominent climate scientist not subscribe 100% to the Romm-Mann view of "never utter any thought that is not helpful to The Cause".....
Many here want to convince Myles Allen of our various arguments on the facts and significance of Climategate etc. That's understandable as a goal, but my more limited hope is that he would take someone like Judith Curry seriously and engage with her world over time.
Wait, you're lecturing on what climategate was all about, and you haven't actually read the emails?
Skiphil, he may have slightly criticized some US political advocate, but he is nowhere near breaking ranks with the UK climate establishment, which seamlessly includes CRU, the Met, The Royal Society, government advisers and so on. He ain't gonna see anything wrong in the emails, because he ain't gonna read them, or admit to having read them. He sings from their songbook, and he is in tune.
Myles Allen writes
But they were not "sufficiently confident". That is the point.
They didn't justify their confidence because they weren't confident.
They knew they couldn't justify using the data as a proxy but they wanted to use it anyway.
That is not science.
That is the scandal.
That they decided to "hide the decline".
That is what Climategate is about.
It's not their confidence but their confidence trick.
Mr Allen,
Thank you for responding.
I'd like to say a couple of things:
As I understand it, the main issue raised by the post 1960 proxy record is not that it was removed in order not to show a spurious fall in temperatures, as you suggest, but that as a means of indicating temperature levels which comes to grief almost as soon as it contacts known reality it proves itself to be so deeply flawed as to be useless. How can it be proposed as an accurate measure for a distant time period?
And since you mention 'spurious two-degree warming' and the surface temperature record, I do not understand how it is that the reduction of temperatures in the thirties and the raising of temperatures post 1950 by as much as 2.5C (from memory) that Rhoda mentioned at the top of this thread has not attracted more attention. Temperature records from Iceland to New Zealand have undergone the same adjustments, a sort of UHI in reverse. Now, I might be persuaded to accept that the fact that many surface temperature readings are taken in cities or airports or near AC outlets or over asphalt does not for some reason introduce spurious warming. Hard to understand, but conceivable. Yet, in fact the raw temperature record is invariably pushed the other way. If there is a plausible explanation for these adjustments, I have not heard it.
Stephen Fox
Let me try to sum up what I’ve got out of the past few days. I suggested in the Communicate2011 talk that the 0.02K revision to the HadCRUT temperature record was the only change to any published dataset used in the detection and attribution of human influence on climate to have resulted from the UEA e-mail affair, and that this was not generally appreciated by the public. I was using this as an example of how things have gone wrong in communicating climate science: this was not a talk about “climategate” per se.
On whether the “only change” statement was strictly correct, Steve McIntyre has pointed out that the e-mails raised new questions with the treatment of paleoclimate records, Judith Curry has observed that these records are needed to check our estimates of internal climate variability and Ross McKitrick has argued that some of the e-mails showed an improper dismissal of his paper on the correspondence between patterns of warming in the instrumental record and patterns of economic development. Many other points have been raised, but I would like to address these three.
I accept Steve’s point that paleoclimate reconstructions continue to evolve and fresh sources of uncertainty continue to emerge, although my impression is that they were evolving anyway before release of the e-mails and would have continued to do so regardless. This continued uncertainty is a key factor making it difficult for scientists like myself, outside the dendroclimatology community, to make use of tree-ring based data. In trying to cope with multiple blog threads simultaneously, I probably went too far in disparaging tree-ring data, for which I apologize to any dendroclimatologists who might be reading these threads. I do believe efforts to reconstruct pre-instrumental climate represent an interesting and worthwhile challenge: my point was simply that many people seem to think it is the main point of climate research, which it is not.
In response to Judith Curry’s point about the need for proxy reconstructions to test model-simulated internal variability, again, this is a question of “it would be nice if only we were able to do so.” In my personal view, the uncertainties and potential biases in the spectrum of variability that must arise from the process of stitching together multiple tree-ring records (many of which have to be individually detrended), and the fact that we know GCM-simulated variability is deficient on the small scales that the trees are responding to, make it difficult to use proxy records to falsify GCM-simulated large-scale variability. If a GCM disagrees with a paleo-record, do we reject the GCM’s internal variability, the forcing data used to drive the GCM, or the paleo-record itself? We do have observations of variability on sub-century timescales through the instrumental record and new products like the 20th century reanalyses: I think, in the short term at least, these potentially provide more information on internal variability than the millennial reconstructions.
Since the key question for attribution is the origins of the surface warming over the past 30 years, that being the only period for which we have direct observations of forcing, it is the spectrum of internal variability on 20-100 year timescales that is essential. Variability on longer timescales is less important for attribution of causes for the current warming trend. I stress this statement applies to surface temperature. Sea level responds on different timescales, making attribution correspondingly harder.
In response to Ross McKitrick’s point (apologies for being slow on this one), I suspect what Phil Jones was referring to in the “no need to calculate a p-value” remark (although you should really ask him) was the danger of over-interpreting chance covariation. The only p-values that mean anything are those that derive from physically-based hypotheses. It is all too easy to find a high p-value from a chance correlation (sunspots and number of Republicans in the US Senate is the classic example). I wasn’t involved at all in that IPCC chapter, but I would be inclined to agree with their assessment that what you were seeing in that paper was an example of such an acausal covariation, for which the p-value of a pattern-correlation is indeed meaningless.
Then there is the much more general point, raised by Lucia, Rhoda and many others, that my talk was misleading, because “climategate” was not about the data at all, but rather about scientific process and the probity of climate scientists. As Mike Hulme observed, “climategate” meant different things to different people: for me, the implications for the instrumental record were all-important, which is why I was castigating the British press for paying far less attention to the fact that the instrumental record got an almost (in deference to Ross) completely clean bill of health than it paid to the initial allegations. There was an interesting side-thread on why the HadCRUT got dragged into this in the first place, to which I don’t have much to add apart from reassuring everyone that I don’t blame the bloggers for this confusion.
Many people have asserted that the main impact of “climategate” is that we can no longer say “trust me, I’m a climate scientist” until we all come out and condemn CRU, Muir-Russell, Oxburgh, etc. “Trust me, I’m a climate scientist” is not a phrase I have ever used, and I hope I never will. I teach a 12-lecture course to our 3rd year physics students (open to the public if anyone is interested) that starts from the premise “Don’t trust climate scientists” – the point being that, as physicists, they should be able to understand the problem for themselves, and not be expected to take the IPCC’s word for it.
The only basis of trust in science is the reproducibility of results. This is why availability of data and model source code is so important: I have always supported open data, although I have also consistently said that I don’t think Freedom of Information requests are the right way to enforce it. Journal editors can and should enforce a simple “disclose or retract” policy if a result is challenged, and almost all of them do: if any don’t, then the solution is to name and shame them, not set up a parallel enforcement system. I also think it is always better to reproduce results from equations (and, where possible, independent models and observations) rather than “auditing” computer code.
Finally, on the “bad for democracy” remark that upset a lot of people. I don’t want to suppress discussion of the Medieval Warm Period, but everything has an opportunity cost. Time spent arguing over paleoclimate research is time not spent on, for example, the merits of the two degree “goal” agreed in Copenhagen and Cancun, with remarkably little scientific justification. Yet whether we aim to limit anthropogenic warming to two, three or four degrees has far bigger implications for climate policy than the existence or otherwise of the Medieval Warm Period. Why is this not a hot topic in the blogosphere?
Ironically, this whole discussion started from a throw-away post by Paul Matthews in a discussion of a lecture I recently gave on whether it would be possible to frame an effective climate mitigation policy that did not extend the reach of the State in the way that cap-and-trade, carbon rationing or geo-engineering clearly will. Paul has apologized, which is much appreciated, but the damage may be done, Paul. If the European Commission decide to impose carbon rationing in 2020 after another record-breaking warm decade, because we spent this past week discussing Myles Allen’s interpretation of climategate (not to mention his, admittedly poor, choice of shirts) rather than coming up with a less intrusive policy alternative, your grandchildren shall know the reason why.
Apologies for cross-posting on various threads: happy for you to pull this out and put it on the top level, or just merge them all into one, or whatever you prefer.