Bishop Hill More on Nature's data policy
Dec 5, 2010 Climate: Jones Climate: Surface Journals Last week I looked at Nature's data policy and was corrected by Eli Rabett on the existence of a formal policy. ER pointed to Nature's advice to authors at the time of Phil Jones' 1990 paper on urban heat islands:
Nature requests authors to deposit sequence and x-ray crystallography data in the databases that exist for this purpose.
As he notes, this paltry sentence doesn't support the idea that there was a formal policy in place requiring authors to make data available. However, Shub Niggurath has been doing some research, and I think his findings put this lone sentence in some perspective, which is quite interesting.
For a start, the sentence in question comes from a guide for authors, a document mainly concerned with the intricacies of style and formatting. To get a broader idea of Nature's approach to data availability, we have to look elsewhere. Useful information comes from an essay by the editor of the time, John Maddox.
Data ownership and access were causing concern in scientific circles, especially in protein biology and genomic reseach. Discussing the suggestion that data availability should be a precondition of publication, the great man opined:
:
This confirms the idea that there was no formal policy, but seems to suggest strongly that Nature felt that authors were under an obligation to make materials available.
Shortly before Jones published his paper, Maddox introduced a new facility for authors - the ability to deposit supplementary information - the extraneous data that supported a paper's findings. As Maddox noted, with larger and larger amounts of data being processed, there was a real risk that important and useful data would be lost...
With that in mind he announced an innovation to address the problem - the journal would host the material and send out copies to scientists who requested it. Although this was directed primarily at molecular biologists, there was no suggestion it was exclusive.
There is no suggestion that this was compulsory - in fact quite the opposite; it seems to have been assumed that scientists would want to make use of the facility.
This would appear to explain the "policy", if I can dignify the words at the top of the post with that term, in a certain amount of perspective.
It all seems very old-fashioned and gentlemanly, with an assumption that everybody in science wants to behave well. One does wonder what Maddox would have thought authors failing to produce data central to a paper's claims.
So Jones was under no obligation to release his data? Not quite: examination of Doug Keenan's correspondence with Nature suggests a different conclusion. Having failed to get the Chinese station data from Jones and Wang, Keenan approached the journal about the possibility of making a materials complaint. While Keenan didn't pursue the matter with the journal, there is no hint in the correspondence that that Nature felt Jones had an option not to deliver up the information Keenan wanted. Indeed, the journal was quite clear that they would admit a materials complaint, provided the request was "reasonable". In the event, they simply said that if Jones no longer had what Keenan wanted there was little they could do to help, but it is clear that they felt that if Jones had them, he should release them.
This series of postings began with by considering confidentiality agreements and their implications for journals. In the case of Jones et al 1990, there were no such agreements in place, and it is clear that the journal felt that Jones was obliged to release his data, formal policy or not. How they would view a refusal on the grounds that confidential information had been used is an interesting question, which takes us quickly into the realms of unreplicable science.
But that's a question for another time.
Shub Niggurath has a related post here.
Reader Comments (12)
"One does wonder what Maddox would have thought authors failing to produce data central to a paper's claims."
One wonders even more what he would have thought of authors actively urging colleagues to destroy data lest it was subject to a FOI request.
Humans, not just the well educated among us, once had a big grab bag full of traits that made the world a fairly decent place to live. One of these little things was an item called "Integrity". The word is now used to refer to many things, but the old meaning as it once applied to Humans has been lost.
If a paper cannot be replicated, for whatever reason, there should be a mechanism to prevent it being cited ad infinitum, let alone published.
Nature et al can puff up their pious position on data availability with "confidentiality agreements" all they like, but when weighed on real events, they have been found wanting.
For the sake of argument, lets assume Jones was innocent and take his stance on face value for a moment. Doesn't that put the reviewers in the dock? How could they replicate Jones paper without the data, how could they even give it a cursory check beyond spell checking?
These hypocrites turn a few days work into a 20yr mission of "truth". I cannot even imagine the total cost, total waste of man hours, let alone how far back they have set climate science, and society as a result.
Only slightly OT - this was posted this morning on The Air Vent -
"Curious said
December 5, 2010 at 8:59 am
For info. re: review – the AMS Council adopted new Guidlines for authors, editors and reviewers on 22 September 2010.
http://www.ametsoc.org/pubs/authorsguide/authorseditorsreviewersguide.pdf
from:
http://www.ametsoc.org/pubs/index.html"
If the "experiment" or "computer run" cannot be repeated how are we going to challenge such obvious nonsenses as "Cold Fusion" or "Global Warming"?
"Increasing global warming hysteria, decreases editorial integrity at Scientific Journals", is the name of a paper that has yet to be written and subjected to pal-review.
No data or code need be submitted, to support the theory, however, snazzy graphics with random data selected from chinese phone directories, make great presentation aids for lazy journalists.
Graphs should always be shown sloping up, when viewed left to right, and colour schemes (for the UK and Europe) shoul be blue for bad and cold, rising to a warm, glowing, politically correct and good, red.
No matter what your skill level, you must have a computer model to back up your theory, even if your only previous experience of modelling was posing for some photos with lumps of firewood, or an Airfix Spitfire with the wings on upside down
Submitted photographs may be "doctored" and merged. A photo showing coastal erosion off East Anglia, with a background dominated by coal fired smoke stacks, and a foreground showing a penguin offering itself as lunch to a starving polar bear, would nail a few cliches in one, and guarantee publication by WWF, Greenpeace, Louise Gray and the IPCC.
Once you are referenced by the IPCC, you become a Lead Author, enabling you to write ever absurd stuff, and get it into the next IPCC report. Grant funding is yours for life, along with free trips abroad to warm places, where eager undergraduates from UEA, Penn State and Stanford will hail you as a hero and role model
Given that data disappears so that "others can't find what is wrong", I think we must go to a politician for advice when it comes to "Scientific Literature".
Trust, but verify --- Ronald Reagan.
I agree with those who basically say if the result can not be reproduced by an truly independent researcher (ie, not a "Team" member), then you have another possible "cold fusion" or "poly water" result and it should not be trusted as anything but speculation.
Perhaps we should require that until replicated, that such research be only reported in JIR
And the more egregious examples should be taken up by the ( Ig)Nobel Prize committee under the auspices of the "Annals of Improbable Research" Google for details.
We're reminded by Eli not to just watch the pea.. keep an eye on the thimbles, too.
My earlier suspicions are thus confirmed. The reference to sequence and x-ray crystallography data was a red herring.
Yup. It was a red herring and a Nature trick.
As I understand it, "Nature" commands diminishing respect in scientific circles - no amount of sophistry can mitigate the damage inflicted by the coining of "Mike's Nature trick". Either it wises up, flicks its editor and makes a fresh start, involving lavish attention to sceptical views, or it will simply wither on the vine. It's already suffering the consequences of a misguided editorial policy. Aren't we taking aim at, and wasting ammunition on, a shot duck?