Bishop Hill Cox and Ince on the scientific method
Dec 19, 2012 Climate: other Pop-sci heroes Brian Cox and Robin Ince have an editorial in the New Statesman. It's about the scientific method and discusses its application to - among other things - climate:
Let us take the politically controversial issue of climate change as an example. Climate scientists make measurements of observable properties of our planet, such as sea surface temperatures and the area of Arctic sea ice. Over many years, these measurements have formed a large data set. The only grounds for arguing with the data would be specific technical issues with the measurements themselves. One could assert that the satellites measuring sea temperatures were not calibrated correctly, or that there was a methodological error in the measurement of the area of the sea ice. Such criticisms are relatively rare. A more common criticism is of the interpretation of the data using computer models.
All models are, by nature, an approximation to reality. But they are the best we can do, given our current understanding and the power of our computers. The important words here are “the best we can do”. There is no other way of predicting the probability of weather in the future. The only legitimate criticisms would be of specific issues with specific models, or of specific inferences drawn from them. It would certainly be wrong to assert that the ensemble of climate models from various research groups around the world encompassed all possible uncertainties about the future, but it is not logical to attack climate science as a whole, because to do so is to attack scientific method.
The timing of this article, coinciding as it does with Nic Lewis's observations about observational estimates of climate sensitivity, couldn't be better. Climatology needs to explain why the scientific method gets reversed in this area.
(I'm on the warpath about ranty comments. Please be nice and on topic)
Reader Comments (70)
Dec 20, 2012 at 7:55 PM | robotech master
They say the same thing. The models are used for analytic purposes. Then the results taken from the models are compared to observations made continually on site for the structural engineers and, for the aerospace engineers, are compared to results of tests employing wind tunnels and other advanced technologies.There is constant interplay between model and observation.
No one would accept an airplane wing until it had been rigorously tested in wind tunnels and other devices that are designed to make it vibrate as it would in flight. After all that work, the prototype aircraft usually show some cracks in the wings after test flights. More engineering, modeling, testing, and flight tests of prototypes are required.
Pity me. When I fly, I can look out the window and determine the fitness of the wing.
Theo - exactly!
In the distant past here at BH I flagged up the work and publications of NAFEMS:
http://www.nafems.org/
IMO how far the climate models are from being fit for purpose (ie developing world economic and social policy) is inexcusable. And in their defence you have the likes of Michael Mann criticising Freeman Dyson with his ignorant utterances:
"I have to wonder if Freeman Dyson will get on an airplane or if he’ll drive a car because a lot of the modern day conveniences of life and a lot of our technological innovations of modern life are based on phenomena so complicated that we need to be able to construct models of them before we deploy that technology."
http://www.scientificamerican.com/podcast/episode.cfm?id=michael-mann-defends-climate-comput-12-01-10
http://en.wikipedia.org/wiki/Freeman_Dyson
The irony of the title for Cox and Ince's nonsense couldn't be stronger!
esmiff
That's curious. I wonder why the greens you encountered in the UK are on the Right, while in the US they tend to be on the Left?
Theo Goodwin
I'm sorry to disappoint you, but a lot of science proceeds on a "best we can do" basis.
Think of Boyle's Law P/V=T. This was derived from experiments with the newly invented air pump and published in 1650. It led to Newcomen's atmospheric engine and Watt's steam engines.
It took another 200 years of research before it became the Ideal Gas Law PV = nRT.
Would you have condemmed Boyle's work as unscientific because it was the "best he could do" at the time?
Climate models are imperfect, but you don't improve them by complaining that they are weak. You work to improve them.
Entropic Man,
"Would you have condemmed Boyle's work as unscientific because it was the "best he could do" at the time?"
If Boyle had made the claim that he had nailed the relationships among pressure, volume, and temperature for all time then, yes, I would have criticized his claims fully. All scientific hypotheses must be subjected to the most rigorous criticisms, especially by the authors.
Today's so-called climate science makes outrageous claims about the relationships between CO2 and planetary temperature. They should be criticized rigorously. In addition, they should be criticized for not being critical of their own work, especially their past work, and their colleagues' work. They constitute a nasty little elite.
There is at least one genuine climate scientist whose work is beyond reproach and who should be taken as the model, the exemplar, by all climate scientists. That man is Roger Pielke, Sr. In his case, one can say that only highly specific criticisms of individual claims that are based on the very best research and evidence need be taken seriously.
Finally, I am sad to say that you have fallen prey to an error that is a badge of honor on the Left. Because there is "some connection" between Boyle's early theorizing and Watt's steam engine, you are willing to give credit to Boyle. Preposterous.
Dec 22, 2012 at 4:44 AM | not banned yet,
Mann hears an analogy that has a nice ring to it and then deploys it without much thought. A couple of years ago, he published an editorial in the NYT in which he compared his "science" to that of a surgeon whose new found technique promised relief for some patients. It never occurred to him that no surgeon worth his salt would claim that his new found technique is a product of science rather than trial and error and ingenuity applied in the cases of some patients who were hopeless because all known techniques had failed.
Theo Goodwyn
"All scientific hypotheses must be subjected to the most rigorous criticisms, especially by the authors.
Today's so-called climate science makes outrageous claims about the relationships between CO2 and planetary temperature. They should be criticized rigorously. In addition, they should be criticized for not being critical of their own work, especially their past work, and their colleagues' work."
If climate scientists are not self-critical, why have they included modifications from AR4 to AR5 such as the reduced emphasis on drought, even when this weakens their case?
I find unthinking dogmatism mainly among the sceptics. Remove the beam from your own eye.
Entropic man
The Nazis ultra conservative, right wing, anti capitalist, environmental agenda has been forgotten
In Europe, there is the Volke Romantic (folk,back to nature) tradition which was a large part of Nazism. It is the basis of the 'local' you hear environmentalist espouse. It still exists in Denmark.
http://www.guardian.co.uk/books/2012/dec/21/patrick-kinglsley-something-puzzling-state-denmark?INTCMP=SRCH
There is also the aristocratic culture of breeding that predates Darwin. Again the Nazis were deeply involved in Eugenics. Blond hair, blue eyes etc.
http://homepage.ntlworld.com/sealed/gw/aristocrats.htm
George Monbiot's article God of the Soil is a modern manifesto of ultra conservatism. Monbiot supports the Welsh National Party (he lives in Wales). I am definitely not saying Monbiot is a Nazi. Nazism died in 1945.
http://www.monbiot.com/archives/2005/03/22/god-of-the-soil/
In America in the 1960s and 70s, there was a large popular movement against industrial pollution, often lead by academics. That was left wing and very successful.
The problem is that modern environmental policies (like AGW) tend to be regressive and right wing.
"If climate scientists are not self-critical, why have they included modifications from AR4 to AR5 such as the reduced emphasis on drought, even when this weakens their case?"
It has been forced upon them by this website, Judith Curry's website, and many others.
"I find unthinking dogmatism mainly among the sceptics. Remove the beam from your own eye."
Of course you do. You are not aware that a model can serve as an analytical tool only and cannot be used for prediction or in any way substitute for genuine physical hypotheses about matters such as cloud behavior. Neither you nor any Warmist can be brought to the light on this matter.
Theo Goodwyn
"You are not aware that a model can serve as an analytical tool only and cannot be used for prediction"
The atmospheric models used for weather forecasting work quite well.
In the longer term models are as good as their data, which is improving.
The sceptic attitude to models is two valued. They assume a model must be perfect or useless, with nothing in between.
The rest of us accept them as imperfect tools with some predictive capability and capable of improvement. They are also recursive, a good way of generating questions for further empirical research.
Entropic - "In the longer term models are as good as their data, which is improving."
Please can you clarify this? Which data and how is it improving? Thanks
PS - please stop putting words in people's mouths with "sceptics" think such and such. I don't believe I have seen anyone express the view "models must be perfect" - have you got a reference? And have you visited Tamsin's blog?
not banned yet, theo goodwyn
Sorry for the delay, I've been coughing green slime since Christmas night.
Thanks for pointing me towards Tamsin. I'm somewhere in the convinced blob on her graph.
http://allmodelsarewrong.com/the-sceptical-compass/
Regarding models, think of them as quantitative hypotheses.You set up a simulation and play "what if" with the variables.The simulator on Roy Spencer's site is a good simple one for this, if you feel the urge to play.
http://www.drroyspencer.com/2010/04/simple-climate-model-release-version-1-0/
Within the limitations of the model you can quickly see which variables have a big impact on the behaviour of the system, and where you need more information. For example, compare AR4 with the leaked draft of the AR5 science paper and you can see the effect of an extra 5 years research. In the Clouds chapter particularly, there's a lot more detailed discussion of cloud effects on the system. It will be interesting to see how that affects the projections of the current generation of models.
How do you validate the models against reality? Aftercasting helps. You plug in past observations for a particular time, add the subsequent changes in CO2, insolation etc. over a subsequent period and compare model output with reality. As with any other finite element model you iterate between model and reality until you get the best match you can.
Starting with current conditions and predicting future conditions is harder. A model which aftercasts well gives you some confidence that it might forecast as well.Unfortunately you cannot validate your predictions by observation until you get there. Alas, those are exactly what the politicans ask for. What can you do? Try to make clear that these are estimates, and give probabilities as far as possible.
Tamsin is correct that all models are wrong. We can, however, get useful estimates of the limits within which they are wrong, and the likely constraints on future conditions.
There's still some missing information.
For example, there's an unidentified energy sink somewhere in the system, which has been soaking up heat since 2003 while air and sea surface temperatures have remained fairly static. This was not predicted by the models.
The imbalance between incoming and outgoing total energy is currently at -1.1 +/- 0.4W/M^2.
http://www.agu.org/pubs/crossref/2009/2009JD012105.shtml
On a total energy input of 5.5 * 10^24 kJ/yr thats, a surplus of 4.02 * 10^21 kJ entering the climate system, and mostly unaccounted for.
By my own calculation 4.06 *10^17 kJ has gone into melting ice sheets and floating ice each year, but the rest is harder to pin down. Warming of the ocean surface and atmosphere can't do it, The Arctic Ocean is warming fast, but not that fast. ENSO averages neutral in recent years.
The only other sinks offhand are the deep ocean, and the permafrost, where our information are still incomplete. We talked about ARGO, and one thing that shows is that the earlier estimate that warming goes only to the thermocline at 700M is an underestimate. In fact some warming takes place to 1400M. (I've mislaid the reference for this. Damn!)
As you know, my bet's on the deep ocean , and possibly the thermohaline circulation. Any other ideas?
Theo Goodwyn
"If climate scientists are not self-critical, why have they included modifications from AR4 to AR5 such as the reduced emphasis on drought, even when this weakens their case?" Entropic man
"It has been forced upon them by this website, Judith Curry's website, and many others." Theo Goodwyn
This intrigued me, so I went looking. The only documented example I could find of a sceptic having a significant influence in the mainstream climate arena was this.
http://data.giss.nasa.gov/gistemp/updates/200708.htm
Perhaps you can give other examples.
Entropic - I note you missed out a reference to support your "two valued sceptic attiude" claim and then put in a lengthy non specifc response on the data that you claim is improving models.
IMO a measure of model improvement would be that the error bands on their predictions would be reducing and the accuracy of their predictions would be improving. The work that springs to mind in this context is that of Demetris Koutsoyiannis and his colleagues:
*******
On the credibility of climate predictions
D. KOUTSOYIANNIS, A. EFSTRATIADIS, N. MAMASSIS & A. CHRISTOFIDES
Abstract
Geographically distributed predictions of future climate, obtained through climate models, are widely used in hydrology and many other disciplines, typically without assessing their reliability. Here we compare the output of various models to temperature and precipitation observations from eight stations with long (over 100 years) records from around the globe. The results show that models perform poorly, even at a climatic (30-year) scale. Thus local model projections cannot be credible, whereas a common argument that models can perform better at larger spatial scales is unsupported.
*********
Unlike the link you provide to AGU it is not paywallled and the full paper can be found here:
http://itia.ntua.gr/en/docinfo/864/
IIRR there is also a good slide set to go with the paper but I don't have time to google it down for you.
The Pielke Snr commentary here speaks to your point about the applicability of GCM to policy:
http://pielkeclimatesci.wordpress.com/2008/07/31/on-the-credibility-of-climate-predictions-by-koutsoyiannis-et-al/
"A fundamental and societally relevant conclusion from this study is that the use of the IPCC model predictions as a basis for policy making is invalid and seriously misleading."
As far as the rest of your post goes - I thought you were in "wait and see" mode re: data on ocean heat uptake rather than putting a bet on it? And as far as my ideas go, I think "wait and see" covers them quite well.
In case Cox and Ince are interested, Lucia is focusing on model output as presented in fig. 9.8 of the AR5 SOD:
http://www.nature.com/ngeo/journal/vaop/ncurrent/extref/ngeo1671-s1.pdf
Sorry, I'll read that again:
http://rankexploits.com/musings/2012/things-i-can-see-in-figure-9-8-of-the-ar5-sod/
not yet banned.
Ok, you win. I accept that no sceptics seek perfect models, they all seem to regard models as useless.
I'm cant get my mind off this missing heat problem. You question regarding how to get warm water below 2000M, and mention of the saline water coming out of the Mediterranean sparked some ideas, for which much thanks.
There's no point thinking aloud any more about it here, so I'm off for a while.
Bye and thank you. See you on other threads.
Entropic Man, under normal circumstances it would be right to try to judge the record by looking for "documented" instances of corrections, etc. However, ClimateScience-TM does not operate by normal, rigorous, fair-minded scientific practice. The attitudes evinced in the Climategate emails are often more like those of Wigley toward "skeptics"....
Climategate-2 emails
Thus, one must begin the discussion of corrections by recognizing that for more than a decade, at least, Mann and friends have expended enormous energies not to acknowledge or credit any corrections from alleged "skeptics" even when the situation was obvious to fair-minded observers. The prevailing method seems to be to make corrections surreptiously or under cover of a new study, and then pretend the field has "moved on" with no assist whatsoever from critics. For instance, Gavin Schmidt went so far as to fabricate a tale of a supposed anonymous submission to the British Antarctic Survery to avoid acknowledging that he had seen and submitted (to BAS) an error identified by the Climate Audit blog of Steve McIntyre.
CA post -- Gavin's Mystery Man Revealed
There are in fact numerous instances of corrections and improvements to the record of temperatures, proxy studies, historical comparisons etc. Just look at how quickly the Gergis et al (2012) study had to be withdrawn and re-done when its actual methodology was exposed at Climate Audit as flawed and in conflict with methods stated in the paper.
CA post -- Gergis et al struggle not to withdraw paper and not to credit assistance from Climate Audit
It is difficult to document this record properly when certain climatologists prove all too willing to obscure, cover-up, and/or fabricate the actual record of scientific criticism, changes, and improvements.
p.s. I hope you are getting over your illness! Happy New Year!
Entropic - did you read the material I linked from Demetris and Lucia?
It is not that sceptics simply regard models as useless: it is the evidence of evaluating their performance that shows them to be useless for the purpose of policy making. Cox and Ince were completely wrong in their support of them - which is particularly ironic when they were commenting in a piece on the "scientific method".
IMO you should study the critical voices on all topics - they might spark some more fresh ideas for you.
Happy New Year and happy investigating.
I despair of Cox. Climate Science is based on the belief that a pyrometer signal is a real energy flux when it is a temperature signal. This is the difference between being a technician, believing the instrument, and a scientist, knowing what the instrument measures. The manufacturers specify you need back to back instruments to measure real energy flow. 100s of man years of data in the 'Energy Budget' are useless....
AlecM
Have you pointed this failing out to Prof Cox? How has he responded?