Following my post on the Royal Meteorological Society's evidence to the AR5 inquiry, Doug McNeall and I had a long and interesting exchange on Twitter. Although he arrived at his point somewhat elliptically, Doug appeared to want to suggest that although in Ed Hawkins' graph the observations are on the cusp of falling outside the envelope described by 90% of model runs, this did not actually represent falsification. In his view, the test was too harsh.
The precise determination of when the observations should be seen as inconsistent with the models is one for the statisticians, and I know that Lucia, for one, disagrees with Doug's view (and I feel pretty sure that Doug Keenan will say that they are both wrong). However, this is not actually germane to my original point, which is that the poor performance of the models to date - as represented by Ed's graph - needs to be communicated to policymakers. We are without doubt less confident than we were that the model ensemble captures the true behaviour of the Earth, even if we are not (in Doug M's view at least) absolutely certain that it does not.
So returning to the Royal Meteorological Society's evidence, I asked Doug why they made the following unqualified statement of confidence in the models (my emphasis):
The Report devotes Chapter 9 to a comprehensive, balanced and realistic evaluation of climate models which is based on the published literature and draws extensively on the results of the Coupled Model Intercomparison Project Phase 5 (CMIP5). As stated in the report (Chapter 9, final draft) climate models are based on physical principles, and they reproduce many important aspects of observed climate. We agree with the report when it states that both these aspects contribute to a “confidence in the models’ suitability for their application in detection and attribution studies and for quantitative future predictions and projections”, and when it notes that “whereas weather and seasonal climate predictions can be regularly verified, climate projections spanning a century or more cannot. This is particularly the case as anthropogenic forcing is driving the climate system toward conditions not previously observed in the instrumental record, and it will always be a limitation.”
I was rather taken aback by Doug's response:
You call that unqualified!? Ha!
To me, the part that I have highlighted seems absolutely to represent a statement of unqualified support for the models. The remainder the merely says that scientists can't say whether they are any good in the very long term. I can only assume that Doug's response is based on a narrow reading of the text - in other words that they are only saying that the basis in physics and the hindcasting ability contribute to a "suitability for prediction", not that such suitability has been achieved. If so I would say that the Royal Meterological Society has grossly misled the inquiry.
But even if this is the case, we know that the models run too hot over the medium term and the short term. We know that they incorporate the wrong value for aerosol forcing, so we would expect them to run too hot anyway. We know that scientists are now theorising that heat is currently being transported into the deep ocean by a process as yet undetermined and entirely unrepresented in the models.
So even if Doug's position is "the models are not yet falsified", we have to ask where is the communication of the known problems with the models. Why has the Royal Meterological Society not explained the situation to politicians?