Bishop Hill +++Acton's Eleven - the response+++
Apr 16, 2010 Climate: Oxburgh Over the course of the day I've made several telephone calls to the Royal Society, without being able to get a response to the simple question of who it was within their ranks who selected the eleven CRU papers for Lord Oxburgh's panel to examine. These papers, you will remember "were selected on the advice of the Royal Society".
Just after 4pm I finally got through to the person responsible and seconds later an emailed response arrived as well. This was fortuitous because I was able to ask some questions on the contents.
So, who was it? Here is the Royal Society's statement:
The Royal Society agreed to suggest to UEA possible members for the Scientific Assessment panel that would investigate the integrity of the research of UEA’s Climatic Research Unit (CRU).
Members of the panel were suggested on the basis of the excellence of their work and their breadth of expertise and experience (including statistical capability).
The Royal Society recommended that the panel had access to any and all papers that it requested and suggested that the review begin by looking at key publications, which were chosen to cover a broad range of subjects over a wide timescale.
Whoah! Get that! They told them only to look at lots of papers. Not which particular ones?
With the press officer still on the line, I asked the question: "So you didn't actually advise UEA which papers they should look at?" and the response was:
"This is the only statement we are willing to make at the moment."
And I said,
"So it's a bit sensitive, then?"
And the reply was,
"This is the only statement we are willing to make at the moment."
So, ladies and gentlemen. I think we can now say with considerable certainty that Lord Oxburgh's declaration that he took the advice of the Royal Society as to which papers to examine is not true.
So who did choose them then?
There's a clue back at the start of the story of Lord Oxburgh's panel. When the investigation was announced, the BBC had this to say:
UEA, in consultation with the Royal Society, has suggested that the panel looks in particular at key publications from the body of CRU's research which were referred to in the university's recent submission to the Parliamentary Science and Technology Committee.
We know, of course, that the bit about the Royal Society is not true. But if you look at the UEA select committee submission, you find in the appendix a list of papers which bears an extraordinary similarity to the list that was looked at by the Oxburgh panel. If you strike off the non-UEA papers, you seem to be left with only a single discrepancy - the appendix points to Jones et al 1997 (J Clim) while the Oxburgh panel looked at Briffa et al 2001 (JGR).
So we have an extraordinary coincidence - that both the UEA submission to the select comittee and Lord Oxburgh's panel independently came up with almost identical lists of papers to look at, and that they independently neglected key papers like Jones 1998 and Osborn and Briffa 2006.
I don't think so. Could it be, could it possibly be that Lord Oxburgh's panel were given a list of papers to look at by UEA?
Acton's Eleven.
Reader Comments (81)
Oh dear - please excuse both my typing and my spelling.
My typing is bad, but I will not hide behind that.
My spelling is very bad, but variable.
Your Grace - do not be distracted,
you are onto something big.
Pull on this string and it may yet all unravel.
Go to it!
Please.
good luck.
This may be defined as the response of the Antidisestablishmentarians to the Disestablishmentarins exposure of the work of the Establishmentarians.
The select selection implies the usual Antidisestablishmentarism response tactic of floccinaucinihilipilification.
With apologies to anyone who suffers from hippopotomonstrosesquipedaliophobia.
I look forward to Booker picking this up for the Telegraph tommorrow
I think all involved should take care when trying to unearth sensitive information of this kind.
After all, people who made FOI requests to CRU are now being questioned by police and "experts in extremism", to see if they might have been involved in releasing the Climategate e-mails.
http://www.telegraph.co.uk/earth/environment/climatechange/7594656/Police-question-climategate-information-seekers.html
Martin Rees and Ralph Cicerone wrote a letter to the Financial Post on 9th April last amongst other things they said:
"It is the responsibility of scientific organisations like ours to present the public and politicians with a balanced assessment of the evidence... "
It doesn't look as though Rees has taken his responsibility seriously to me.
Welcome Judy, as ever a balanced view, with a position we can all start from. You maybe right about there being no malfeasance, but I'm not sure, "malfeasance" in climate science seems to have a very narrow definition. Read the Bishop's book on the Hockeystick and the subsequent attempts to cover up the work and ask yourself whether future generations of scientists will view it with the same tolerance.
As for the eminent scientists, I believe they are taking the climate science community on it's own assessment. That they believe it isn't possible for scientists to engage in malfeasance and that therefore, like all juries should do, came to the table with a presumption of innocence (like Judy). Since no evidence was provided that could possibly have found the CRU wanting in scientific integrity and given the probability that the never spend any of their time looking at sceptical blogs and their assumptions on the sceptics are that they are a bunch of swivel eyed, foam flecked, creationists in the pay of big oil, and you have the perfect recipe for no one wanting to blacken the name of an academic colleague.
Keep up the good work Andrew, but I fear that Prof Rees will prove an elusive prey. All he has to do is to keep reiterating his satisfaction with the report until you, (a) find the killer evidence, or (b) go away. I think he's betting on (b).
As a minor aside, I wonder if Judith could take a moment to answer a question. I've posed the same question to another research scientist today - not a climatologist, but an entirely different discipline, and he's a supporter of precautionary measures against the perceived threat of AGW - and though I haven't had a response yet, I'm fairly sure I know what it will be.
Judith, would YOU ACCEPT an assessment of the integrity of YOUR work, conducted in the same way Oxburgh has conducted his assessment of the CRU's?
I realise it's a loaded question. I realise it may even be perceived as entirely rhetorical. And I suppose that's probably even my point; it surely has to be a rhetorical question.
Response entirely optional, Judith; no presumptions either way.
As a small follow-up to my minor point, I wonder if Judith expects that the CRU staff, particularly Prof. Jones, will accept - perhaps even be proud or pleased - by Oxburgh's level of detailed examination and consequent exoneration of the body of CRU's work?
I don't know what Scientistfortruth's beef with Martin Rees is, but I think he's being rather over the top and intemperate in his wording. In his subject areas Rees has produced very worthwhile work, and I've read his books and some papers which repay the effort. As Judith says, the processes have been corrupted, if there is anything more personal that SfT percieves it probably does not belong here IMHO.
@poptech
The correspondence has been asked under the Environmental Information Regulations (which is kind of like a FOIA) by McIntyre.
Amen to the excellent Judy Curry's post. Although on balance she is a Warmer - perhaps less so these days - she has always had the respect of the hard-bitten sceptics at Climate Audit (Steve McIntyre is studiously neutral, except when he finds misconduct).
Yeah, starting again from scratch is the only way forward, and if the process trashes a few ill-gotten reputations, it can be a warning to the future.
Hats off to Judith for a very balanced response, she puts the other Climatologists to shame.
Dr. Curry's summary of the panel process strikes me as largely on target - though she is probably too forgiving on the obligations of scientists to avoid issues of confirmation bias. Isn't that the message of Feynman's delicious piece on Cargo Cult Science?
"the idea is to try to give all of the information to help others to judge the value of your contribution; not just the information that leads to judgment in one particular direction or
another.
http://www.lhup.edu/~DSIMANEK/cargocul.htm
What makes matter worse, IMHO, is that the panel also seems to have adopted the approach of the South Seas Islanders in their assessment of the scientific work of CRU researchers.
"The RS ... suggested that the review begin by looking at key publications..." could be taken to mean either that the RS submitted a specific list of suggested key publications, or merely that the RS suggested that the review compile its own list of key publications.
So it's not crystal clear to me, at least, that the RS did not submit a list of suggested publications.
In the U.S., the National Science Foundation recognizes three types of scientific misconduct:
Fabrication – the actual making up of research data and (the intent of) publishing them, sometimes referred to as "drylabbing".
Falsification – manipulation of research data and processes in order to reflect or prevent a certain result.
Plagiarism – the act of taking credit (or attempting to take credit) for the work of another.
At most, the CRU activities might fall under "falsification," but this refers to laboratory data and controlled experiments, rather than the rather messy situation of observing nature
Wikipedia has quite a good article on scientific misconduct:
http://en.wikipedia.org/wiki/Scientific_misconduct
(any quotes below are from the wikipedia article)
"This is distinguishable from other concepts such as bad science, junk science or pseudoscience where the criticism centres on the methodology or underlying assumptions. It may be possible in some cases to use statistical methods to show that the datasets offered in relation to a given field are incomplete. However this may simply reflect the existence of real-world restrictions on researchers without justifying more sinister conclusions."
The more accurate description of the climategate activities are arguably this:
"Suppression/nonpublication of data: A related issue concerns the deliberate suppression, failure to publish, or selective release of the findings of scientific studies. Such cases may not be strictly definable as scientific misconduct as the deliberate falsification of results is not present. However, in such cases the intent may nevertheless be to deliberately deceive. Studies may be suppressed or remain unpublished because the findings are perceived to undermine the commercial, political or other interests of the sponsoring agent or because they fail to support the ideological goals of the researcher. Examples include the failure to publish studies if they demonstrate the harm of a new drug, or truthfully publishing the benefits of a treatment while omitting harmful side-effects."
So I still don't think that misconduct has occurred, rather we are seeing possibly "bad" science and suppression/nonpublication of data. Both are very undesirable particularly with regards to high impact and policy relevant science.
The allegations seem to be more of the bad science, suppression of data type. Re the Bishop's book, I haven't read it yet (but i am ordering it this weekend). I would certainly be interested in seeing if people have actual indications of formal misconduct.
I'm not sure, Judith. I think that appending observed temperatures in place of proxy temperatures may amount to falsification as defined by the US NSF. Exclusion of unsupportive data would also amount to falsification.
If one understands falsification to refer to the result of the research (as I do, reading the definition you posted), rather than falsification of the data underlying it (else it would run up against the fabrication clause), I think there may well be a case to answer.
** exclusion of unsupportive data would be data manipulation, if supportive segments of the same data series were included.
When asked to make a clear and concise statement which reflected the panels view of the outcome of the enquiry, a spokesman for the royal society offered the following :- "I think that it is safe to say that things are clearer now than they were before they were as clear as they are now"
He went on to declare that he felt able to say this without fear of contradiction.
Whether the RS itself chose the list of publications, and if so how, is clearly a key question, and a simple one for the RS to answer, if they have nothing to hide. The most natural contruction of the key passage in their statement - 'which were chosen' - is that the RS itself made a choice of key papers (as a non-binding suggestion for the committee to follow). If they were proposing to the committe that the committee should make its own choice, without suggestions from the RS, the natural formulation would be something like 'which should be chosen', not 'which were chosen'. And yet the RS spokesman dodged the direct question "So you didn't actually advise UEA which papers they should look at?" It is this dodging the question which makes me think they have something to hide. The phrase 'which were chosen' is ambiguous, since it is not explicit about who did the choosing. This may be because they want to conceal some involvement of the UEA and/or Phil Jones in the selection process, but there is also another possibility: the RS may in fact have suggested a list which the committee did *not* follow completely, e.g. it may have included some of the controversial papers but the committee may have decided *not* to examine these. This is all just speculation, but as an ex-bureaucrat I am scratching my head to think of some reason why the RS should put itself in such an awkward position, first by making an ambiguous statement, and then refusing to clarify it. Contrary to some perceptions, bureaucrats don't actually *like* stonewalling reasonable questions, and they usually do so only in order to conceal some underlying disagreement, e.g. between their own organisation and an 'ally'.
Juddith curry
I think that you would struggle to argue that the jury in this enquiry was not rigged. You only need to reg a jury when expedience is the order of the day. They (the enquiry panel) knew therefore that the facts of the case would not bear close scrutiny took the steps necessary to hide them. This means that they are fully aware that something is amiss. They know it, we know it and we know that they know it. In my opinion, either the UEA or the enquiry panel, or both, are guilty of malpractice.
pologies for the errors and bad grammar (Mixing tenses etc)
Judith Curry - thanks for posting and engaging with these issues. Where do you think the abuse of the peer review process falls within the NSF definitions? Would this come under "processes" in their definition of falsification - it seems to have been done to exclude results which certain scientists did not want to acknowledge?:
Falsification – manipulation of research data and processes in order to reflect or prevent a certain result.
I echo the thanks of the many posters for Judith's participation.
'The corruptions of the IPCC process, and the question of corruption (or at least inappropriate torquing) of the actual science by the IPCC process, is the key issue.'
'inappropriate torquing'- brilliant. The English language can always produce the right gem.
Ironic quote of the day
'I wonder whether any other generation has seen such astounding revolutions of data and values as those through which we have lived. Scarcely anything material or established which I was brought up to believe was permanent and vital, has lasted. Everything I was sure or taught to be sure was impossible, has happened.'
My Early Life: A Roving Commission (1930), Chapter 5 (The Fourth Hussars) Winston Churchill
Nice Bishop! "And I said: 'So it's a bit sensitive, then?' " ;-)
Real Climate has spotted the Curry response, it starts here:
http://www.realclimate.org/?comments_popup=3846#comment-171153
Frank O'Dwyer is being strangely silent about all this.
Mottos:
"Nullius in Verba"
Obviously no one at the Royal Society understands what that now means.
also there is
"Do Different"
No change at UEA then.
"...suggested that the review begin by looking at key publications, which were chosen to cover a broad range of subjects over a wide timescale"
Considering that Jones, alone, has over 50 papers to his credit, the sampled 11 strikes me as being somewhat akin to the methodolgy used in the selection of data for the hockey-stick (or use of one weather-station's temperatures as proxy for another 1,500 km away!) It's a "methodology" that's obviously been quite successful for them in the past :-)
On the same day that Oxburgh's "report" was released, the results of a "Citizen Audit" - Donna Laframboise's crowd-sourcing project - were also released. As one of the auditors, I can attest to the fact that each of the 18,531 References in the 44 Chapters of AR4 was examined by 3 independent auditors - in order to determine whether or not Pachaur's claim
http://www.noconsensus.org/ipcc-audit/not-as-advertised.php
(echoed ad nauseam by the media and other CAGW proponents) that the IPCC Assessment Reports are "all/solely/entirely/only" based on "peer-reviewed" literature is supported by the evidence in the references.
21 of the 44 Chapters earned an "F" (fewer than 60% of the references cited were to material published in peer reviewed journals).
Christopher Booker recently mentioned this audit in his column. AFAIK, to date, this is the only MSM coverage given to this audit. Yet I can well imagine that if we had used sampling similar to Oxburgh's, we probably would have received considerable MSM coverage ... and they'd be shredding our study to pieces!
http://hro001.wordpress.com/2010/04/14/uns-climate-bible-gets-21-fs-on-report-card/
Is it possible that The Oxburgh report (pdf) has been removed from the UEA web site?
http://www.uea.ac.uk/mac/comm/media/press/CRUstatements/Report+of+the+Science+Assessment+Panel
@hro001
That "audit" is for the most part an enormous, biased straw man.
The IPCC *does not* use solely peer-reviewed literature for every single thing, so this is a non-issue for starters.
Not only that, but the "audit" gives failing marks on a number of spurious instances. For example sections where the IPCC explains how, say, the current report differs from previous ones, providing citations for the previous reports - guess what, those count as fails. Or where well-established scientific publications *that predate peer-review* are referenced - well, they fail too.
This is precisely the kind of time-wasting nonsense that does nothing to advance public understanding. It is misinforms, rather than clarifies. It is childish, "gotcha" point-scoring.
@Dave H
"The IPCC *does not* use solely peer-reviewed literature for every single thing,"
Quite so, as our audit confirmed. This being the case, perhaps you should ask yourself why Pachauri brandishes "it's all peer reviewed" (and/or variants thereof) like a crucifix before vampires whenever anyone has the temerity to criticize or question the IPCC Assessment Reports.
It's also worth noting that the IPCC "rules" specify that non-peer-reviewed material is to be designated as such in the References. Of the 5,587 such references (i.e. those found not to have been published in peer-reviewed journals), only 6 were so designated.
Notwithstanding your opinion that the audit was mere "time-wasting nonsense" because, well ... because you said so, I suppose ... the purpose of the audit was to debunk the myth of IPCC infallibility perpetrated by Pachauri - and echoed by a multitude of media mavens who failed to perform any due diligence before parotting his claims.
hro001
So if you want to combat misleading statements by the head of the IPCC, why not use a link from the IPCC website that clearly states non-peer-reviewed literature is used? Simpler and more effective, surely?
http://www.ipcc.ch/publications_and_data/publications_and_data.htm
And a simple question - in your opinion, is providing a citation to a previous IPCC summary report when explaining differences in the latest revision allowable under IPCC rules for references? Yes, or no?
Judith, hro001;
Who decided which were the "key" publications, and which specific papers were to be examined?
As for exclusion, if a drug trial excludes all the "data points" of fatal side-effects, and concludes the drug is safe, this is falsification with a capital F.
hro001; The average grade in the audit works out to "D". Most of the "F" ratings are fatal, in any case. A chain of evidence is as weak as its weakest gap or link.