Sunday
Jul032011
by 
Bishop Hill
Bishop Hill Material World on climate models
Jul 3, 2011 Climate: Models The BBC's Material World programme interviewed Prof Paul Valdes, a climate modeller. The message appears to be that climate models are very bad at reconstructing major climate shifts in the geological record and are probably bad at predicting future ones too.
The conclusion of the interview appears to be that it's worse than we thought. This struck me as slightly odd given that the rest of the interview appeared to revolve around the fact that the models don't tell us anything very useful.
Eduardo Zorita has further thoughts at Klimazwiebel.
Reader Comments (80)
Z,
I don't think anyone here has a problem with using models to "fill in" current climate/weather conditions. Is that what you've been getting at all along?
Well that's an argument over the meaning of words. What do you mean by "analysis"? Models are entirely legitimate for analysing "what if?" scenarios. But one has to remember that the particular "what if?" does not mean the problem has been correctly encapsulated as it applies to the real world; only that problems of the kind, "if this is how it works, what if?" can be explored. Moreover, the output of any such model is guaranteed to diverge from reality because initial conditions are estimates and precision is limited. For both of these reasons models are tools not evidence.
I think the main issue I have with models is the press release that accompanies the output of the latest run. It's almost always, "according to Climate Scientists, AGW could cause X to happen". The public misunderstand what "could" means here, and how models actually work.
"I don't see anyone changing the subject here, just people confused as to what exactly you are talking about."
It's OK, JJB, we're used to it.
Mine years ago was that the modelers were trying to keep their little trains on circular tracks on the ceiling.
===================
Eagle-eyed Revkin
Peered, and drew an honest sword;
God bless him and save.
=============
Jeremy, one other way of evaluating a model when there is no truly out-of-sample data (because the whole data set has been used in developing the model) is to use "cross-validation". In its simplest form this would mean splitting the data set into two, training the model on the first set, fixing the parameters and then running it on the second set; then repeating the process with the data sets swapped, so that now the model is trained on the second set and tested on the first. The quality of the model is then assessed by looking at how well it performs on the two sets of data when it have been trained on the other half - so that there is no benefit of being tested on exactly the data it have been trained on. This is by no means perfect, but when the size of the data sets is small in comparison with the number of parameters involved in the model, this may be quite sufficient to show that the predictive power of the model is near zero.
I should like to see this sort of methodology applied to climate models, which I suspect are massively over-parameterized
Zed you know full well that you are just a spoiler, a pain in the ass. You dont believe anything you just like throwing a spanner into the works now and again.
YOU seem to confuse graphical representation of facts with models which try to predict things we dont have all the facts for.
Monitoring global climate requires only instruments which record temperature. Recording those measurements is most easily done with visuals like graphs but none of this requires models. Maybe you should revisit your A level file notes mate.
ZDB
A model predicts and instrumentation monitors as can be seen by clicking the link. Thank you for the opportunity to point out both your and the IPCC deficiencies.
Nicholas, sure, I know about cross-validation. That being said, I don't think it would work here, because the modellers would claim that they do not 'train' and certainly not 'fit' their models. They would claim that there is no 'training set'. Sceptics would claim that the whole time series is some kind of 'tuning set' - the people developing the models are so aware of what the temperature series looks like that they cannot ignore it when choosing the parameters for their models (hence the introduction of the deus ex machina of aerosol). It is difficult to find a middle ground here. Arguably, the models have been tested in this way, and failed - before including changes in aerosols in the post-war period, they fit the recent trend in temperature but not the 'test set' of earlier temperature trends.
The point I was trying to make, though, is that the space of observations is far wider than just the one-dimensional time series of yearly global average temperature anomaly (GATA), and that provides some testing beyond just temperature. On such wider tests, models do much less well than on GATA. Because that kind of data is typically spatially resolved, it also produces such a broad set of data that it is much harder to imagine doing 'tuning' of parameters in order to fit it. There are also issues about whether models *should* be able to fit it, given the stochastic nature of models, but that is another matter.
Note the subtle shift of meaning between one post and another. She starts off by suggesting that you can monitor the climate with models then when those who understand English call her on that she moves on to her usual tactic of simply being offensive.
What precisely, Zed, is this "rich seam of real ignorance" you've just exposed, because for the life of me I can't see it and supposed to be moderately bright (I'm told)?
Model output is not empirical evidence and there's no "technically correct" about it. You're splitting hairs and playing with words again, just as you always do. Given good enough models one can perhaps come to some broad predictions about possible courses that climate might take provided they have been accurately programmed with sufficient variables. But you have to know what these variables are before you do the programming so models are no use for telling you what has happened in the past and since the climate is a chaotic system they are only a little help when it comes to foretelling the future.
In neither case are they "monitoring" anything.
So. Which red herring are you lining up for us next?
ZDB,
Or this.
A note on what "real" science is like, from the BBC.
The team must weep when they read this stuff.
http://www.bbc.co.uk/news/science-environment-13988836
Well Z, it's been fun, but I have to close with a most excellent Ad Hominem:
Apparently, skeptics are smarter.
Remember, this is scientific evidence produced by scientists using actual data, so it couldn't possibly be open to question :p.
Folks. Can't we all just ignore Zebedee and let her go away and grow up a while?
ZBD. Come back when you are a bit older. There's a good girl.
Jul 4, 2011 at 1:03 PM | Alexander K
I agree that the quest more more complicated models on faster computers is to a large extent just wanting new grown-up toys to play with. Models due have some use when used correctly, ie they are a very useful aid in weather forecasting, but it is useful to also have an experienced meteorologist on hand to apply his experience to any forecast. Also a forecasting model is only as good as the initial data that you put into it. I think there is more of a push to throw money at the model rather than at 'boring observations' as that is the way to get your new toy to play with. I think in the weather forecasting world some of the observation networks are getting worse through time due to this and more reliance on satellites for input data.
Models are also a useful tool for paleo-climate research and it probably is worth using them to investigate things like sudden climate changes. But as the article here shows, currently they are pretty useless and representing what the Earth really does.
From New Scientist, frogs thought extinct found "under our noses all the time", but of course
"They are worse off than we thought" . LOL
http://www.newscientist.com/article/mg21028173.600-extinct-frog-was-under-our-noses-all-the-time.html
Das
Damn formatting!+wrong button
Obviously it wasn't being monitored well enough. Or more likely, somebody's model said that it must be extinct and so it was assumed that it was.That's the problem, Zed. If you stick your nose too far into models then you end up not being able to see what's right under it and that applies to weather and climate even more than to frogs.
Sorry about that!
@zed
What seam of ignorance have you exposed? I honestly can't see it, and would genuinely feel enlightened by an answer that didn't involve a volley of mild insult. All you seem to have exposed is your own misunderstanding of the function of models (unfortunately shared by most of the climate modelling community), and of what the term 'monitoring' means.
I read the comments here a lot, but rarely post. Commenters seem to be educated, rational and articulate, with varying backgrounds, from interested layperson to scientist to academic. I have also read many of your posts, including ones where you accuse other commenters of ignorance, yet fail to demonstrate any understanding of the points you bring up, or wider knowledge of the scientific basis of your claims. You often argue against what you wish people had said, rather than what they have actually said, and you often attempt to rebut rebuttals by claiming a different position to the one you started with. Your arguments are routed with little effort, and your contradictions and errors are often ridiculed, which I think is sometimes unfair (as they are always entertaining). This doesn't put you off coming back for more though, which is something I admire, yet find baffling.
There is obviously something strong that draws you here, beyond the desire to bring people round to your point of view (which is something you appear to have little interest in). Perhaps there is a true sceptic in you struggling to get out, but ruthlessly denied for fear of betraying your political and social identity - kept in tightly check by your prejudices, your ideas of what it means to be an advocate of a 'noble cause', and to be a 'denier'? Perhaps you read the latest alarmist press releases, ready made rebuttals, hollow caricatures of sceptics, abstracts of flimsy 'team' papers (see Mann's latest sea level effort), and there is a nagging voice in your head telling you that it just doesn't add up. You loathe commenters here because you really want to be among them, using your intellect for analysis and criticism rather than propagating logical fallacy, but can't as you fear it would mean abandoning a cause that gives you a sense of purpose. Still, by posting and taunting, you can get close enough, yet at the same time maintain your identity. If this is the case, then maybe you need to hear that it's okay to believe in rationality, scepticism, the scientific method, and at the same time hold a strong social, environmental and political conscience. Maybe you'll be happier embracing your inner sceptic rather than railing against it all the time..
;)
OMG, Zed's been on the toot again - I was always told never to argue with a drunk.
I think Zedzee should consider the possibility that he/she may be on the wrong side of this thing.
Let Freedom Ring
Andrew
May I just step in here and defend Miss Z by making an assumption of good faith on her part.
I too groaned at the usage of the term "thermometers" used in a particular context earlier. The Earth is an open system with energy entering and exiting the system continuously. The residual energy which keeps the Earth warmer than it would otherwise be is called the energy balance.
My metaphor for the energy balance is a cup of cappuccino - the froth is the surface air temperature and the liquid is the ocean. The substantial part of the energy budget is held in the oceans.
So we're dealing with energy entering and leaving, surface air temperatures and ocean retention of energy in order to determine the energy budget. That requires modelling as the information is incomplete even with satellite information hence Trenchberth's frustrations. We give it our best guesstimate.
The biggest problem facing everyone in this debate is that the climate is a bound chaotic system. Chaotic systems are stable but can shift without any external influence. And we can't predict chaotic systems which is the elephant in the room regarding predictive models.
Z is not the only one getting confused by monitoring and models, in this case the divergence does not invalidate the theory.
http://wattsupwiththat.com/2011/07/04/a-peer-reviewed-admission-that-global-surface-temperatures-did-not-rise-dr-david-whitehouse-on-the-pnas-paper-kaufmann-et-al-2011/#comment-694108
Given the widely noted increase in the warming effects of rising greenhouse gas concentrations, it has been unclear why global surface temperatures did not rise between 1998 and 2008.
But in the conclusion:
The finding that the recent hiatus in warming is driven largely by natural factors does not contradict the hypothesis: “most of the observed increase in global average temperature since the mid 20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations (14).”
When will they finally admitt AGW is a failed theory !!!!!
Jeremy, Nicholas
It is impossible that climate modellers are not aware of the global temperature series, and even though they don't explicitly tune for it, there is no way they are not aware of it and decisions made will inevitably be coloured by this.
Of course, it is possible to test the model output by increasing the number of degrees of freedom either spatially or temporally, or use something other than the first moment of the time series. But tests conducted on this basis simply fail every time.
We're then told its not fair, models only work globally and at the 30-year timescale, so we are left with no choice but to wait.
But I can say, with high confidence, there is no reason to expect the current batch of models will do any better than Hansen's failed 1988 predictions. Paul Valdes identifies one of the reasons why not in this paper. We have another modelling approach that does capture the geological scale problems, but it does not get a mention because it does not toe the IPCC party line.
Z,
The best question that you've ever asked is "Are We there yet?"
Quit when you're ahead mate. There are some pretty clued up people on this forum. Excluding us two, of course.
Best Wishes, your wee mate RoyFOMR
XXX
Jul 4, 2011 at 9:29 PM | Spence_UK
Is there a best model for the weather on Earth since the last glacial maximum??? I am definitely interested in which 'simple' model that seems to work and what it thinks is important in defining our current weather.
Knock knock.
"Anybody home?"
Knock Knock
"Are you hiding under the bed, Zed?"
Long silence...
Footsteps, fading into the distance.
Don't knock too loud, hangover you know !!!
The problem with the models is surely the funding - it is political not scientific - and so it is attracting the least scientific as they are most able to bend the rules.
Half of the models are missing - because that half is not understood. As a computer engineer I find it completely bizarre that anybody would be prepared to trust their long term predictions.
If a model is unable to predict a change in direction it is clearly missing key drivers. If those key drivers are missing from the model, how can anybody state with any certainty that you are using the correct drivers at any other time? They are presumably iteratively updating a dataset. The more iterations from known data the larger the error.
At this stage the testing of the models should surely be to see if they are able to explain minutae of the climate system better than non-model based predictions.
I have trouble understanding how anybody with a scientific background can believe that a model with as much missing as these models have are useful for long term predictions. A model with excess positive feedback will always predict that the system it is modelling will go to hell in a handcart. That effect will drown out everything else in the model after a relatively short number of iterations.
As Larry says the key point when using a model is to identify the key drivers as he puts it. So surely the best approach currently to modelling is to simplify the models down to what you believe these key drivers are. It also has the massive advantage of being much, much cheaper computationally. Once you have a simple model that is believed to represent the important processes correctly then maybe throw in extra complexity to see if you can get better detail. Currently adding more and more complexity without the key processes included will probably just throw out spurious correlations for what you are looking for.