Bishop Hill Liz Wager on conflicted peer review
Feb 10, 2011 Climate: Surface Journals There has been an interesting discussion in the Steig thread about whether Eric Steig should have been invited to be one of the reviewers of the O'Donnell paper or not. On the one had there is the fact that Steig, being the subject of the critique, had a conflict of interest. On the other, he would have been the person best able to point out possible flaws in the O'Donnell paper. Opinion among commenters appeared divided. With this in mind I wrote to Liz Wager at the Committee on Publication Ethics - an advisory body for scientific journals - to ask for her thoughts. Here they are:
Should an author whose work is the subject of a criticism in a submitted manuscript be among the invited peer reviewers of the manuscript?
COPE (the Committee on Publication Ethics) doesn't issue general guidance or proscriptions on how peer review should be done but we do mention criticisms in our Code of Conduct for editors, namely
"Cogent criticisms of published work should be published unless editors have convincing reasons why they cannot be. Authors of criticised material should be given the opportunity to respond.
Studies that challenge previous work published in the journal should be given an especially sympathetic hearing."
In developing this guidance, we had in mind letters to the editor rather than new analyses/papers but recognise that practices differ in different areas (apparently maths journals never print correspondence so if you want to criticise another person's work you have to write a new paper). So COPE says that the authors should be given the opportunity to respond to specific criticism of their work, but we do not provide guidance about whether they should peer review papers criticising their research. We leave that up to the editor.
Liz Wager, Chair COPE
Reader Comments (106)
woodentop
You are right of course, but it is dangerous to allow the focus of interest to move to J Climate. Too much long grass there.
Hence my insistence that we remain clear that the actual 'offences' against O'Donnell et al. were hobbling and slowing the review process and strategic damage done to the paper itself.
I don't 'care' as much about what J Climate did. The insight into Team behaviour is Steig's behaviour.
It's also a real poke in the eye for the legions of apologists who swear up and down that the Team doesn't do this kind of thing and is being smeared by contrarians playing climate politics.
BBD
I did not miss the point. This blog posting by BH is about anonymous reviews in general and how they apply to the Steig/O'Donnell issue.
I simply wanted to make it clear that (i) the JoC behaved appropriately in allowing Steig to be a reviewer even if he was anonymous (ii) the JoC editor stepped in and effectively killed Steig's contribution when it became apparent that a lot of his complaints were not material/trivial and clearly beginning to look like an attempt to delay publication. I am not sure to what extent this was caused by the letter to the editor by O'Donnell. I have not looked in detail at the correspondance although I have looked through all of reviewer A's comments.
I have no wish to defend Steig. He clearly used a method which was flawed and we wait to see how he defends it against O'Donnell's most recent sensitivity plots. Our focus should in my view be on the science and the methods. He also seemed to push things pretty far in his reviewers comments which were getting quite ridiculous by the time the editor stepped in. I wonder what he would have done if the JoC had had a practise of non-anonymous reviews. My guess is that Steig would have done exactly the same. There is too much to lose for him after the fanfare of his initial paper.
Fred Bloggs
Fair enough. You did not say this in your earlier comment.
Viv is correct (Feb 10, 2011 at 6:57 PM | Unregistered CommenterViv Evans) in that all too many of those commenting both here and on other sites are not paying sufficient attention to the way in which the pea is being shuffled such that we are all in danger of losing track of the real issue: O10 demonstrated that S09's statistics were incompetent. The arguments over conflict of interest, should JoC have brought Steig in as an anonymous reviewer, was Steig, in fact, a front man for a "team" review, etc., are all secondary issues. The primary issue as BH demonstrated in an earlier post making O'Donnell's CA post more readily understandable, is that S09's statistical methods were incompetent. All of the other points raised, while worthwhile, esp. as the "team's" attempts at suborning the peer review process have been exposed in broad daylight for all to see, are secondary. Please return the focus to where it belongs.
Climate "science" and its statistical modelers desperately need collaborators from those who are well versed in modeling chaotic systems. Those who study computational fluid dynamics come to mind as a particularly well qualified group. One may, of course, draw one's own conclusions from that fact that the climate "science" community is woefully lacking in statistical competence and that its various GCMs are murky (cloudy?) at best as has been demonstrated repeatedly at least since MM05 and that they have done nothing but "circle the wagons."
As SM keeps repeating, follow the pea and don't allow yourself to become distracted by secondary issues.
"...The 'hockey team' are indeed a team, and will 'protect their own'." --David C
With friends like that, who needs enemies?
"Before Climategate the Team would have got away with this." --Dr John
This has been an educational experience for a lot of people, including, perhaps, editors of journals.
"I am not conversant with the statistical nuances of the analyses by Steig et al. and the approach adopted here, so trust that Eric Steig or Michael Mann will provide that needed expertise!" --Reviewer B
BWAHAHAHAHA!!!!
I think Don's analogy is also worth repeating:
I stick to my comment that Steig clearly had the right to give his side of the story, but from the witness box and not the jury box. None of this would have happened if the editor had kept that in mind, particularly with regard to a critical paper that basically tears apart previously published work.
There's another key reason why Steig should not have been asked to review the O'Donnel paper; he's admitted himself that he is not an expert in statistics.
RayG
Good comment.
As you know I am simply trying to keep the focus on Steig's (mis)behaviour as a reviewer, but I willingly concede that you are making a larger and more important point. Just glancing up, I would say that jorge and lapogus probably agree.
The observation was made earlier (by Thinking Scientist) that not all of the 88 pages were from Reviewer A; that was the total length of the conversation. Somewhere (sorry, I forget where), it was stated that 24 pages were review; the remaining 64 would then be responses to review comments.
Still comes out as OTT in my book.
P.S. The reason I write "Reviewer A" rather than Dr. Steig is that I reserve judgment on whether Steig wrote all 24 pages of review comments. It is mere speculation, nothing more.
timheyes
@ Don Pablo. I wasn't arguing with you - just sticking my 2 cents in.
No, you weren't and I know it. I was making a point about the use of "review". As ThinkingScientist pointed out, changing one word in what you said made it much clearer. In fact you and I are almost completely in agreement. It was just semantics, that's all.
And please continue to contribute your 2 cents or whatever you want. This is a DISCUSSION, not a DOGMA INDOCTRINATION. Your opinion matters, even if others disagree. That is what the fun is all about.
All BH asks is that you are well behaved, and you certainly have been.
I even find ZDB sometimes makes an interesting point or two when she behaves.
woodentop
That was down to Reviewer A being in the position he was. Once he got into that position, the rest would have been a predictable possibility (or probability, depending on your level of cynicism).
So was it rape or prostitution? :)
@BBD
I don't believe I am focusing on J Climate. I'm focusing on Reviewer A, his position and indeed what (predictably) followed, however he got there, although his acceptance of the position in the first place was egregious enough in my opinion.
If you had added "and the apparently unethical abuse of the position he was in." to the end of that sentence we would be in accord,
This incident is a perfect example of pal (or in this case foe/faux?) review and the damage it causes. That's the mischief that needs to be addressed in the peer review system as interpreted by the climate "scientists", in order to prevent similar car crashes occurring in future.
I'm not holding my breath.
@RayG - you're quite right - the science is, self-evidently, an important issue. However the other side of the coin is that the Team and assorted hangers-on like directing the argument off down highly technical nitpicking cul-de-sacs, boring most passing observers into confusion and / or apathy.
It's important to keep in mind that a more general point which applies right across the field is the reliance on what is described as "peer-reviewed science". Every time that's shown to be a facade, it removes another prop from the rickety structure of climate science.
@DPdlS - per Sting's famous (apocryphal?) remarks about his love life, I'd describe it as "tantric peer review".
woodentop
Baffled. I'm fairly sure that what you feel is missing from the earlier comment:
Is what I said:
I'm very tired, and perhaps I'm not reading you right. But I think we are in accord.
@BBD - we differ only in emphasis on where the problem lies. But it results in the same thing: the perversion of science and, in the case of "climate science", politics too.
KnR said:
Journals and their processes should remain a matter for those journals. Variation is how things evolve. A standardised rulebook just makes it easier to cheat and to do so everywhere. Journals should be more transparent and the scientific community (so far as there is one) should be willing to hold journals to account.
A regulated journal system would always benefit those at the top of the pile (for they would be lauded as the model for lesser journals to ape) yet it is sometimes those most illustrious of publications that fail to be circumspect and instead choose the avenue of advocacy.
There is a much broader issue here which seems to have fallen through the cracks - the lack of temperature recording stations.
The essence of the disagreement between S09 and O10 is the fairly heroic and seemingly inappropriate statistical methods used to calculate temperature trends in places where there are no thermometers.
If only a small fraction of the money spent on AGW was diverted to better and more measuring stations then all of these ugly extrapolation and interpolation issues would just go away.
Perhaps I missed it, but I never ever hear calls for this sort of stuff. All I hear about is the need for supercomputers.
@Gareth - fine. As long as such policies are known beforehand and everything is out in the open.
From recent postings it would seem that peer review panels can be constituted as follows:
* the journal picks respected, independent (as far as possible) experts in the field;
* contributors can suggest reviewers to the journal;
* journals can pick (anonymous) reviewers whose papers are being criticised;
* a criticised author can be given a right of reply
* a mixture of some/all of the above
If the selection criteria were out in the open an observer could make a judgement about how much reliance could be placed on the journal's published (and unpublished) papers.
don Pablo, woodentop, the way I've always thought about it, in peer review, the Editor is the judge - and the referees are the witnesses. There isn't a jury - the judge decides on his own.
About where the focus or the story should be, or is, my view is that there are so many interesting angles here - that's why it is so fascinating and everyone in the blogosphere's writing about it. The reason I waded in (a lurker till now) is because of one of the angles. This story shows beautifully how the Team's claims that peer review has some kind of magical virtues are bunkum. Peer review is the worst system for pre-selecting papers for publication - except for all the other methods. It leads to bad results quite often through people's misplaced instincts, their wrong-headedness, their negative gut feelings about their opponents, their positive gut feelings about their allies, and their sheer stupidity. You don't even have to bring deliberate misconduct in for things to go wrong (though that does happen sometimes also). Most practicing scientists know all of this. One side of this is that lots of really bad papers (or papers which turn out to be bad) get published, even in high-profile journals, e.g. MMR/Wakefield (Lancet) or 'Arsenic Life' (Science, just recently) - or Steig 09. The other side is that other papers get bogged down in the review process, end up getting mangled, or published in less high profile journals than they deserved, or delayed - or just rejected. It happens all the time, even in fields where the only thing at stake is a few people's vanities. In this field, there's so much emotional and political baggage invested in it on both sides, but especially by the Team, that all the dysfunctional aspects are in overdrive. That aspect seems a good point to try to get over to people - peer review combined with groupthink and people's prejudices can lead to erroneous ideas getting a lot of traction for a long time.
Perhaps a way to satisfy the transparency and conflict of interest issues is that, during the review process the reviewers remain anonymous but on publication their identities should be revealed.
I see no problem with Steig as a reviewer, as many have said he should know he is conflicted and act according, the editor should know he is conflicted and act accordingly.
It is clear that Steig's ability to act as a scientist and not engage in a defense of his own paper during this refereeing process was judged as sufficiently inadequate by the editors of a highly regarded climate science journal that they replaced him.
A fascinating contrast would be the referee notes for S09 from Nature. Did anyone spot the fundamental statistical flaws, or was it just waved through because its message was convenient.
J - that is an extremely good description of how what is publicly held to be a rational and objective process actually works when dependent upon the goodwill and co-operation of your actual human beings. Peer review is (or was intended to be) just a first screen, a filter that takes out coarse/major errors and obvious lack of quality. The real test of a paper should then be the reactions to it, and its longevity. In the current bunfight it's become very clear that Steig '09 was never worth the front cover postions it achieved, and is now more or less a ragged piece of windblown litter. O'Donnell et al will have a firmer postion, but only in that it showed weakness in the former paper (which was all it was intended to do anyway). In the sense of science then, O10 beats S09. In terms of short term publicity and influence? Probably S09 still has its nose in front.
What has clouded the issue in climate science rather more than in other fields is that the Peer Review has become the tool of the Press Release and the 'team' are more or less honour bound to prevent any opinion counter to that of their paymasters achieving the badge of honour. From that point of view, Climategate and the current excitement have made it plain that the peer review process is much less authoritative than has been claimed, and as a propaganda tool it is now much less useful. It can return to being a useful screeing tool for which the imperfect procedures elucidated here, and on other blogs, are probably adequate. Hopefully that means that a lot more young and enthusiastic scientific enquirers (not 'scientists' necessarily) will have the opportunity to be published.
An afterthought about referee anonymity: timheyes and others were saying it was alright to have Steig give input, but not anonymously. Well, the editor knows who the referee is. So the latter is not anonymous in that sense. To use don Pablo's analogy again - the judge knows if the witness is the accused's best friend (or the victim's).
Apart from allowing Steig to excessively draw out the review process, I think J of C got it just about right.
As for Steig, Schmidt, Mann and the team, duplicity and obfuscation are their scientific method.
'....I still hold to that, but there is evidence of improved progress: Steig shows promise, doesn't he?' -BBD
I don't know.
I don't know how many times Stig and his co-drivers have driven this race previously.
Maybe they no longer need practice laps.
Tony Hansen
Check for irony ;-)
Extract from CRU email string 1254259645.txt
From: Michael Mann <mann@xxxxxxxxx.xxx>
To: Andrew Revkin
Subject: Re: mcintyre's latest....
Date: Tue, 29 Sep 2009 17:27:xxx xxxx xxxx
'if McIntyre had a legitimate point, he would submit a comment to the journal in
question. of course, the last time he tried that (w/ our '98 article in Nature), his
comment was rejected. For all of the noise and bluster about the Steig et al Antarctic
warming, its now nearing a year and nothing has been submitted. So more likely he won't
submit for peer-reviewed scrutiny, or if it does get his criticism "published" it will
be in the discredited contrarian home journal "Energy and Environment". I'm sure you
are aware that McIntyre and his ilk realize they no longer need to get their crap
published in legitimate journals. All they have to do is put it up on their blog, and
the contrarian noise machine kicks into gear, pretty soon Druge, Rush Limbaugh, Glenn
Beck and their ilk (in this case, The Telegraph were already on it this morning) are
parroting the claims. And based on what? some guy w/ no credentials, dubious connections
with the energy industry, and who hasn't submitted his claims to the scrutiny of peer
review.
Fortunately, the prestige press doesn't fall for this sort of stuff, right?'
@ ThinkingScientist
You make some fair points. As far as 5) goes, I thought this was normal practice but only if the editor agreed and knew who the other reviewer was. (Usually because some part of the paper deal in an area where the reviewer was nto expert).
I'm holding off believing that the paper was reviewed by The Team and comments from The Team were returned to Nature via Steig until there is some evidence for this. Currently all I've seen is speculation but circumstantially there is possibility that this may have happened
RE: TimeHeyes
Just to restate it, my comment 5) was: To tell the editor that you are going to pass the paper to someone else because you do not have competence on the statistics should not normally be allowed - the editor would want to know who, and if you have a recommendation the editor should normally contact them as an extra or alternate reviewer directly.
I have once been in a similar position to this. A very controversial paper was submitted to a Soil Science journal suggesting a link between soil type and childhood diseases. Such a topic would be highly controversial, especially if picked up by the press. One reviewer was a very competent statistician who I had done some work with. Because he knew I had additional special knowledge in geostatistics but had also studied soil science at university he passed it to me for any comments that I could think of. He had obtained permission from the editor to do this, but I was not contacted directly by the editor.
So I have to accept that it can happen as you describe, but I think it is unusual. If I am asked to review a paper and I take one look and think "that's out of my expertise" I usually just decline. I must stress, though, I am not a big reviewer - I probably get asked to review 3 - 5 times a year and sometimes decline just because I don't have the time. Depends how interesting the paper looks. Also, the journals I review for are generally set up completely on-line - after accepting the review, you download the paper, uplaod comments on-line and never really talk directly to anyone.
Pharos
Heh.
You couldn't make it up.
timheyes
I agree. It's paranoid speculation on the thinnest of evidence.
timheyes
Sorry, should have asked:
"comments from The Team were returned to Nature via Steig"
I think we are talking about Journal of Climate rather than Nature?
WUWT has a post up on this:
http://wattsupwiththat.com/2011/02/10/reviewer-a-responds/
@ThinkingScientist
I defer to your knowledge on this. I did a science degree but don't work in research and have never had any involvement in scientific publishing or peer review.
@BBD
You're correct. My brain failure.
I note this comment by Andrew Revkin today
I am intrigued, having followed the current controversy, as to what O'Donnell may be planning to apologise for. Can anyone enlighten me, or is Revkin just blowing hot air again?
@ Peter Wilson
I don't know specifically what O'Donnell may need to apologise for but an equally interesting question is "who is copying Revkin on emails and why?".
Woodentop
@DPdlS - per Sting's famous (apocryphal?) remarks about his love life, I'd describe it as "tantric peer review".
Well played, sir. Well played. Game, set and match.
j
It is a complex issue, and it depends on the editor. Some are strong willed, others not. Generally, in my experience while in academia, the reviewers do have a definitive vote. And much as a judge can overrule a jury decision, so can the editor. But even editors are subject to peer review and peer pressure. They can and have lost their jobs much like a judge can be recalled.
Generally, the paper is refused if a sizable proportion of the reviewers think it is worthless, and sent for revision if a number of reviewers feel it needs improvement. The editor can, and does exert executive privilege, but rarely does.
I might also point that, apparently except in Climate Science, reviewers are also reviewed. You are given a paper to review by an responsible editor because she or he has faith in your knowledge and analytic abilities and you do your best to be totally professional in doing that review. And if you do not live up to expectations, your performance soon becomes general knowledge and you reputation suffers. Academia is a very small world, and everybody knows everyone in a particular field. You can not afford to be a flake -- with the apparent exception of Climate Science, whatever that is.
In any case Steig should have had the opportunity to respond to the paper, and in no case should he have had a vote. His concerns should have been forwarded to the reviewers for their consideration. Judging by the comments of others above, It would appear that Reviewer B felt that Steig was going to have input and was awaiting that input. Indeed, had I been a reviewer of that paper, I would have requested the editor to forward the Steig's comments for my consideration. That appears never to have been done. Or at least so we are told. Now, what actually happened through the private back channel communications is unknown. And I will leave that point there as it is not germane.
I do fault the editor severely: Steig should have not have been on the jury. I believe my analogy is quite accurate. It really is like a court case and the rules should have been followed by the editor. And I seriously doubt if Ryan ever expected that his paper would not be shown to Steig. It was the proper thing to have done.
And I will repeat my point that Ryan O'Donnell has acted like a gentleman through this whole ordeal, and is being very diplomatic even to day. Steig clearly crossed a line in his actions throughout this and deserves to sit on Josh's Stool of Shame.
Don Pablo De La Sierra writes:
"Steig clearly had the right to give his side of the story, but from the witness box and not the jury box."
Very well said. I think my take on the matter is similar to yours. Being a reviewer means being an impartial judge. Being an impartial judge means having no personal interest in the fate of the essay that you are judging. However, the essay of O’Donnell’s that Steig judged was a criticism of Steig’s work. Given that fact, Steig should have said to the editor that I have a conflict of interest. If the editor replied that it does not matter, then Steig had a responsibility to ask if O’Donnell agreed. We know that these questions were not answered because O’Donnell did not know that Steig was a reviewer. Therefore Steig had a conflict of interest. Steig had a conflict of interest and was morally wrong not to reveal it to all other interested parties.
Is it possible that Steig-gate was the only solution that Gavin could dream up to solve the "Steig et al 2009 Has Been Debunked' problem?
Rather than having to defend and then retract an incorrect alarmist propaganda piece from Nature, the debate is forever transformed to a discussion of peer-review ethics.
If true, the ruse has been remarkably successful. No one is worrying about the science now. Even Steig has said that he may not bother with a rebuttal, and not one eyebrow twitched at Real Climate over this Steig comment. All journalists will see of the false Steig paper will be the subsequent peer review scandal, not the original faulty propaganda piece.
Gavin is, without a doubt, a genius.
There does not need to be evidence of conspiracy when you are looking at the work and publications of like-minded people.
For those who have not been following the http://wattsupwiththat.com/2011/02/10/reviewer-a-responds/ thread over at Anthony's. Here are my edited highlights from last night:
jorgekafkazar says:
February 10, 2011 at 11:46 am
Sir Arthur Conan Doyle believed in fairies. When others claimed to see fairies, he believed them. He thought the Cottingley fake photographs likely showed fairies.
The Team believe that adding CO2 to the atmosphere increases the greenhouse effect, producing higher temperatures. Since there’s no way to detect that process other than indirect measurements, they pursue novel methods to tease a warming signal out of some of the noisiest data on the planet. If those methods don’t show warming, they tweak the methods until they do. If the data don’t show warming, they tweak the data until they do. There simply must be fairies!
They are so certain there are fairies, they see fairies wherever they look. If you don’t see the fairies, they’ll provide the Cottingley photographs, and the printout from their Global Fairy Models, too. Do you believe in fairies yet?
Dave Wendt says:
February 10, 2011 at 11:47 am
Although this ongoing controversy has provided some valuable insights into the many flaws in the peer reviewed publication process for scientific literature, it has seemed to me from the beginning to be a colossal waste of time and effort. The temperature data from Antarctica has always been so sparse and unreliable that even if, by some unimaginable miracle, you could get the entire statistical community to agree on a set of “best practices” on how to deal with it, you still couldn’t produce anything more meaningful than a statement that Antarctica is colder than a Wiccan’s mammary and is likely to stay that way, no matter what global mean temperatures do in the next couple of centuries.
As I tried to point out when Steig’s work first surfaced, this effort is analogous to trying to construct long term temperature trends for the continentalU.S. by taking data from stations in the Florida Keys and the Everglades, combining it with data for the other 47 states from a handful of randomly scattered stations which were some of the worst rated in Anthony’s surface station survey, and producing a nicely scarlet swathed map of the lower 48. The only real difference is that Antarctica is actually twice the size of the lower 48.
I realize that dedicated number crunchers have a psychological need to believe in the efficacy of their efforts. A realistic assessment of the true uncertainties of their efforts would likely subject them to profound existential doubts about the meaningfulness of their life’s work. The rest of us shouldn’t feel compelled to share their delusions.
And Phil has just played a stormer, so is worth repeating here. If what Phil contends is correct, it kind of blows a very big hole in S09, irrespective of the flawed methodology.
======================================================
http://wattsupwiththat.com/2011/02/10/reviewer-a-responds/
Phil says:
February 10, 2011 at 9:33 pm
From Response to Third Review A, page 8:
…the reviewer seems to misunderstand the difference between spatial and serial [auto]correlation.
From http://climateaudit.org/2011/02/07/eric-steigs-trick/#comment-254159:
Steig stated by email today that he did not see the Response to Reviewer A’s Third Review…
It seems to me that Dr. Steig should not have launched into a public criticism of O’Donnell et al., when he, admittedly, was not a statistician and apparently did not understand a subtle, but key, statistical distinction.
Futhermore, it should be emphasized that Steig, et al. have not been completely transparent with regard to Steig 2009. They initially promised to electronically publish “all” of their data, but, subsequently, they have withheld and are continuing to withhold the raw satellite data as well as the details of the cloud masking. I don’t think that this data will ever be disclosed, because, if they do disclose it, IMO any remaining credibility would vanish. However, I could be wrong. Here is why I think that Steig et al. may need to be withdrawn:
1. Steig et al. claim that their study is based on the satellite data (NATURE| Vol 457|22 January 2009, pg 462):
We use passive infrared brightness measurements (TIR) from the Advanced Very High Resolution Radiometer (AVHRR), which are continuous beginning January 1982 and constitute the most spatially complete Antarctic temperature data set. (emphasis and acronym added)
They reference: Comiso, J. C. Variability and trends in Antarctic surface temperatures from in situ and satellite infrared measurements. J. Clim. 13, 1674–1696 (2000) (Comiso 2000) and state as the only explicit change to Comiso 2000:
We have updated the data throughout 2006, using an enhanced cloud-masking technique to give better fidelity with existing occupied and automatic weather station data. We make use of the cloud masking in (Comiso 2000) but impose an additional restriction that requires that daily anomalies be within a threshold of ±10 °C of climatology, a conservative technique that will tend to damp extreme values and, hence, minimize trends. (parenthetical comment added)
2. Comiso 2000 states:
Except for 1979 and 1992, when infrared data for the entire year were processed, the derived temperature data that are currently available are only for a winter (July) and a summer (January) month during (1978-1997). (parenthetical comment added)
References to seasonal trends are made in O’Donnell 2010, Steig’s first review (Review A) and its response, in Review C and its response and in Review D. In no way should this comment be taken as a criticism of O’Donnell 2010 as the processed AVHRR data was provided by Dr. Steig without further clarification and without any gaps (Steig et al Antarctica ant_recon.txt is what I had downloaded – it may have a different name now). The reader should be cautioned that Steig et al may have processed the infrared data for all months of the year and for each year, instead of what Comiso 2000 states, but, given the reference to Comiso 2000 and the refusal to disclose the raw satellite data, this question is not frivolous.
3. Steig 2009 makes no mention of how much AVHRR data is lost due to cloud masking. Kato et al. 2006 (S. Kato, N. G. Loeb, P. Minnis, J. A. Francis, T. P. Charlock, D. A. Rutan, E. E. Clothiaux, and S. Sun-Mack, Seasonal and interannual variations of top-of-atmosphere irradiance and cloud cover over polar regions derived from the CERES data set, GRL, VOL. 33, L19804, doi:10.1029/2006GL026685, 2006) states (pg 3):
The mean cloud cover over Antarctica is relatively constant, ranging between 0.62 and 0.75 during all seasons.
Thus, it can be assumed that only 25% to 38% of the AVHRR data is retained after cloud masking. Furthermore, Comiso 2000 states as one of their conclusions:
Among the key results of this study are the following: (a) satellite infrared data provide spatially detailed maps of surface temperature in the Antarctic region with an accuracy of 3°C…
Thus, each datum of the satellite data matrix provided by Dr. Steig should have an individual uncertainty probably greater than 3°C, but certainly in whole degrees C, yet Steig 2009 asserts that there is statistically significant warming by calculating trends in tenths of degrees C with confidence intervals expressed in hundredths of degrees C. I would submit that such an assertion is highly questionable and should be believed only upon rigorous demonstration. (Once again, this is no reflection on O’Donnell 2010 as they specifically state in the response to Reviewer D that:
Because our expertise is with the mathematics, we prefer to limit our paper to the mathematics.
)
Given all of the above, I respectfully submit that Steig 2009 should be withdrawn in its totality as the claimed warming trends appear to be a fantasy, given the cloud masking data losses of about two thirds and the uncertainty for each remaining datum of at least 3°C according to Comiso 2000 (Dr. Comiso is a co-author of Steig 2009). I remain open to withdrawing this comment if Steig et al. can rigorously demonstrate that the total uncertainty of the satellite data doesn’t swamp the small warming trends that they claim (I would likewise say that any claim that there is a small cooling trend would also be a fantasy on the same grounds). Withdrawal of Steig 2009, however, IMO should not impact O’Donnell 2010 as they have conclusively demonstrated that the statistical methodology employed in Steig 2009 is erroneous and that result would survive in any event. In closing, I would like to congratulate the O’Donnell 2010 authors for their fine scholarship.
My only comment on O’Donnell 2010 is that they should have included somewhere the response to Reviewer D’s comment on pages 1-3 of said response that the O’Donnell trend was not statistically different than the Steig 2009 trend, as this may be a common misunderstanding when comparing two different trends. I thought the explanation by O’Donnell et al. was very instructive and it shouldn’t be buried in a review response.
Hockey Team 0 - O'Donnel et al 2
Peter Wilson,
I suspect RO will apologise for the tone of his post, which is the kind of things adults do...which I further suspect the clowns ar RC will latch on to as an example of how wrong he is, even though RO isn't saying sorry for being wrong.
Assuming that these claims are true of course and army a figment of ES's imagination (sorta like how a lot of ES's data seems to be a figment of his imagination :)
Regards
Mailman
@BBD
'I once suggested that the Team weren't much good at strategic thinking'
You and me both, buddy.
I've been wondering for some while over at Judith's exactly what they are trying to achieve. Because I can see no discernible 'winning' strategy in their actions.
Having spent some years where their word was Law and they had only to pronounce upon a topic for the MSM and other commentators to eagerly lap up their offerings as Gospel Truth, they have completely failed to adjust to the changing world around them.
Whereas once the line 'I;m a climate scientist expert - you must listen to me' - received dutiful deference from all parties, this is no longer the case. The 'barbarians' are at the gate, and don't fall in line quite so reverently any more.
Given this - i.e the opposition have turned up and are in play for the first time in a long time, it is utterly bonkers to continue with the same strategy.
To issue mountain top pronouncement and then disappear into their own echo chamber allows the sceptics a free run over the places of public influence in the blogosphere - and increasingly in the MSM as well.
Their influence over journals and other publications can only be weakened by shenanigans like the Stig and O'Donnell. First to show that Steig was substantially wrong and guilty of believing in an artefact of the maths, not of the data (a problem the good Bish is very familiar with :-) ). And second to be caught gaming the system. Bad marks all round.
But their biggest error is their longest standing one. By a blog policy of heavy moderation, they give neutral observers the impression that they have something to hide. When many blogs (this is a good example), thrive with only a light touch on the tiller and allow debate to go where it will, their insistence on rigid orthodoxy to the the team line suggests that either they do not understand the concerns of many outside of their little coterie (score nul points for intellect) or they are afraid to debate them because they fear they will lose (score nul points for their science).
Strategically they are like Hitler about to declare war on the USA or Napoleon invading Russia in the autumn. Both extremely dumb and pointless things to do- but both following from their own misguided convictions that they are invincible.
It will all end in tears. Here's another guy with a failure of strategic thinking suddenly realising that the game is up
http://www.youtube.com/watch?v=GU53qv5aA1M
lapogus
Thanks for the repost of 'Phil's' comment on WUWT. Really very interesting.
Lord Oxburgh
Generally agreed especially WRT the moderation policy at RC.
They wouldn't last long in my world ;-)
ZT
Gavin is, without a doubt, a genius.
I find that when some diabolically clever stratagem is claimed to have lead to a brilliantly contrived result, it was usually due to abject incompetence and bungling. I see no reason to not believe the same is true here.
And yes, I did see the </sarc> :)
Don Pablo:
My aunt is a genius...and she is also very sensible.
one thing i have noticed in reading all the various posts until my eyes bled.o,donnel et al paid for this out of their own pockets.there was also a restriction of around 8,000 words i believe.maybe 9, 000.my point is that by delaying publication there was the possibility of stopping the paper through shortage of funds.failing that constant rewrites meaning the baby being thrown out with the bathwater which would seem to be the case with the section on Chladni patterns .it seems that alone would have destroyed S(09).so despite all the other shenanigins.the team got what they wanted.given the current situation i dont think it would be feasible to think any reveiwer would be beyond their reach maybe im wrong but in light of a comment above it would seem the reveiwers knew each other .is that normal i would have thought anonimity covered all participants with only the editor knowing all the interested parties
Don Pablo, if you're still reading, you're right that neither Steig nor the editor performed with the utmost level of integrity... And you may be right that allowing criticisees to referee criticising papers has more disadvantages than advantages, and hence should not be done. But it does happen, commonly, in many fields of science, so I don't think that what happened to O'Donnell's paper was abnormal practice dreamed up by the editor specially for this paper as a way to gatekeep it. The problem in this field seems to be that a hell of a lot of people have a high level of flakiness, which is so far inbuilt that they have no idea that it is there.