Bishop Hill Questions to ministers
Apr 9, 2013 Climate: Statistics Climate: Surface This is a guest post by Doug Keenan
Questions relating to the work of the Met Office on global warming are being put in the UK parliament, and the Met Office is refusing to answer them. Parliamentary Questions have a history going back centuries. Giving answers, or giving a valid reason for not answering, is required. The stand-off is yet to be resolved.
The Parliamentary Question that started this was put by Lord Donoughue on 8 November 2012. The Question is as follows.
To ask Her Majesty’s Government … whether they consider a rise in global temperature of 0.8 degrees Celsius since 1880 to be significant. [HL3050]
The Answer claimed that “the temperature rise since about 1880 is statistically significant”. This means that the temperature rise could not be reasonably attributed to natural random variation—i.e. global warming is real.
In statistics, significance can only be determined via a statistical model. As a simple example, suppose that we toss a coin 10 times and get heads each time. Here are two possible explanations.
- Explanation 1: the coin is a trick coin, with a head on each side.
- Explanation 2: the coin is a fair coin, and it came up heads every time just by chance.
(Other explanations are possible, of course.)
Intuitively, getting heads 10 out of 10 times is very implausible. If we have only those two explanations to consider, we would conclude that Explanation 1 is far more likely than Explanation 2.
A statistician would call each explanation a “statistical model” (roughly). Using statistics, it could then be shown that Explanation 1 is about a thousand times more likely than Explanation 2; that is, statistical analysis allows us to quantify how much more likely one explanation (model) is than the other. In strict statistical terminology, the conclusion would be stated like this: “the relative likelihood of Model 2 with respect to Model 1 is 0.001”.
A proper Answer to the above Parliamentary Question must not only state Yes or No, it must also specify what statistical model was used to determine significance. The Answer does indeed specify a statistical model, at least to some extent. It states that they used a “linear trend” and that the “statistical model used allows for persistence in departures using an autoregressive process”.
If you are unfamiliar with trending autoregressive processes, that does not matter here. What is important is that HM Government recognized, in its Answer, that some statistical model must be specified. There is, however, still something missing: is the choice of statistical model reasonable? Might there be other, more likely, statistical models?
(There is also a minor ambiguity in the Answer, because there many types of autoregressive processes. A related Question, from 3 December 2012, effectively resolved the ambiguity. The Answer to the Question stated that “Linear trends … are based on year-to-year variability around trends described as autoregressive (AR1) processes” [HL3706]. Other Answers, discussed below, confirmed that the process was of the first order.)
I found out about the Question (HL3050) put by Lord Donoughue via the Bishop Hill post “Parliamentarians do statistical significance”. I then discussed the choice of statistical model with Lord Donoughue. I pointed out that there were other models that had a far greater likelihood than the trending autoregressive model used by the Answer. In other words, the basis for the Answer to the Question was untenable.
Moreover, I had published an op-ed piece discussing this, and related issues, in the Wall Street Journal, on 5 April 2011. The op-ed piece includes a technical supplement, which describes one other statistical model in particular: a driftless ARIMA(3,1,0) model (again, unfamiliarity with the model does not matter here). The supplement demonstrates that the likelihood of the driftless model is about 1000 times that of the trending autoregressive model. Thus the model used by HM Government should be rejected, in favor of the driftless model. With the driftless model, however, the rise in temperatures since 1880 is not significant. In other words, the correct Answer to the Question (HL3050) might be No.
Lord Donoughue then tabled a Parliamentary Question asking HM Government for their assessment of the likelihood of the trending autoregressive model relative to the driftless model. HM Government did not answer. Lord Donoughue then asked a second time. They did not answer. He asked a third time. Again they did not answer. He asked a fourth time. Still they did not answer. He has now asked a fifth time. The answer is due by April 12th.
A Parliamentary Question that has been tabled in the House of Lords is formally answered by HM Government as a whole. In practice, HM Government assigns the Question to a relevant ministry or department. In our case, the Questions have been assigned to the Department of Energy and Climate Change; the designated minister is the Parliamentary Under Secretary of State, Baroness Verma. Verma obtains answers from the Met Office. The person at the Met Office with final authority is the Chief Executive Officer, John Hirst. In practice, Hirst delegates authority to the Chief Scientist at the Met Office, Julia Slingo. Thus, it is actually Slingo who is refusing to answer the Parliamentary Questions, with Hirst and Verma backing her (perhaps without thinking).
I have had a few e-mail exchanges with Slingo in the past. Slingo has never really addressed the issues that I raised, but instead replied largely with rhetoric and a display of gross ignorance about undergraduate-level statistics. For an example, see the Bishop Hill post “Climate correspondents”. Hence, I decided that trying to talk directly with Slingo about the Parliamentary Questions would be a waste of time. Instead, I tried talking with Hirst. I first e-mailed Hirst about this after the third refusal to answer the question from Lord Donoughue. The message included the following.
Last week, Lord Donoughue tabled Parliamentary Question HL6132, about statistical models of global temperature data. HL6132 is essentially the same as HL5359, which the Met Office refused to answer. The Met Office Chief Scientist does not have the statistical skills required to answer the Question; there is, however, at least one scientist at the Met Office who does have the skills—Doug McNeall. I ask you to ensure that the Question is answered.
Doug McNeall is a statistician. He and I have had cordial e-mail discussions in the past. In particular, after my op-ed piece in WSJ appeared, on 12 August 2011, McNeall sent me an e-mail stating that the trending autoregressive model is “simply inadequate”. Indeed, that would be obvious to anyone who has studied statistical time series at the undergraduate level. Note that this implies that a statistician at the Met Office has stated that the Answer given to the original Parliamentary Question (HL3050) is unfounded.
Hirst’s reply to my message was sent after the fourth refusal to answer the question, on 28 March 2013. It is as follows.
I would like to assure you that the Met Office has not refused to answer any questions. The questions you refer to were answered by Baroness Verma, Parliamentary Under-Secretary of State at the Department of Energy and Climate Change.
I note that in her response to HL5359 and HL6132, and a number of other questions from Lord Donoughue, Baroness Verma has offered for him to meet officials to discuss this and related matters in more detail.
My rejoinder is below.
I do not know whether your message is serious or just your way of telling me to get lost. In case of the former, some elaboration follows.
The question that Lord Donoughue has been asking requires the calculation of a single number. The calculation is purely arithmetical: there is no opinion or judgment involved (nor is background in climate needed). Furthermore, the calculation is easy enough that it could be done in minutes, by someone with the appropriate statistical skills. You could think of it as being similar to finding the total of a column of integers.
The number that Lord Donoughue is asking for is 0.001, according to my calculation. (Yes, it is that simple.) Lord Donoughue, though, would like the number calculated by an official body. He therefore tabled Parliamentary Questions asking HM Government for the number.
Lord Donoughue has now received Written Answers to four such Parliamentary Questions: HL4414, HL5031, HL5359, HL6132. None of those Answers give the number. Instead, the Answers make excuses as to why the number is not given. The main excuse seems to be that the number is not important. The importance of the number, however, is a separate issue: even if the number has no importance at all, the arithmetical calculation can still be done, and the number can still be given.
HM Government has been relying upon the Met Office, to supply them with the number; the Met Office has refused to do this. In other words, the Met Office has refused to answer the question—contrary to the claim in your message. What reason does the Met Office have for refusing to supply the number? The required time would be less than the amount of time that the Met Office has spent in refusing.
Parliamentary Questions have a history going back centuries. I do not have expertise in this area, but it is my understanding that HM Government is obliged to either provide an Answer to a Question or else give a valid reason for not providing an Answer. The refusal of the Met Office to supply the number would thus seem to be leading to a violation of a centuries-old parliamentary convention. Indeed, I have now talked with other members of the House of Lords and the Commons about this: there is real concern, and apparently also by parliamentary officials.
Lord Donoughue has now asked for the number a fifth time. The tabled Question is as follows (HL6620).
To ask Her Majesty’s Government … whether they will ensure that their assessment of [the number] is published in the Official Report; and, if not, why not.
The Answer is due by April 12th. My hope is that if the Met Office continues to refuse to supply the number, HM Government will get the number from elsewhere.
I have not received a reply to that. Additionally, I have been informed that Hirst is away this week and next; so there will be no reply from Hirst before the due date of April 12th.
The Met Office is obviously being highly obstructionist. The alternative, though, would be for the Met Office to admit that they do not have a statistical model that supports their claim that the temperature increase since 1880 is statistically significant. In other words, the alternative is for the Met Office to admit that the temperature increase might be reasonably attributed to natural random variation.
It is not only the Met Office that has adopted a position like this. The IPCC, in its most-recent Assessment Report (2007), used the same statistical model as the Met Office. The Assessment Report discusses the choice of model in Volume I, Appendix 3.A. The Appendix correctly acknowledges that, concerning statistical significance, “the results depend on the statistical model used”.
What justification does the Appendix give for choosing the trending autoregressive model? None. In other words, the model used by the IPCC is just adopted by proclamation. Science is supposed to be based on evidence and logic. The failure of the IPCC to present any evidence or logic to support its choice of model is a serious violation of basic scientific principles—indeed, it means that what the IPCC has done is not science. The failure implies that the claim that temperatures have been significantly increasing is unfounded.
Thus, the Parliamentary Questions tabled by Lord Donoughue undermine the primary basis for global-warming alarmism. The Met Office is trying for a cover up. In doing so, it is potentially risking a conflict with parliament.
It remains to be seen how matters get resolved. Under the rules of parliament, the person delegated with responsibility for a Parliamentary Question is the government minister who delivers the Answer. In our case, that minister is Baroness Verma. According to the Companion to the Standing Orders and Guide to the Proceedings of the House of Lords, §4.68 Ministerial Responsibility, “Ministers should be as open as possible with Parliament, refusing to provide information only when disclosure would not be in the public interest” and “Ministers who knowingly mislead Parliament will be expected to offer their resignation to the Prime Minister”.
Bishop Hill
Doug McNeall tweets that this article is relevant to the points Doug Keenan makes.
Reader Comments (69)
Brilliant report of a vital initiative. Thank you Doug.
Over to you, Richard (Betts).
Brilliant write up, even I could understand it! I really hope David Rose or Dellars sees this and it goes far and wide.
Excellent work Doug. It is this persistence, as demonstrated by many other worthies (McIntyre, Swords, Holland, Montford, and many others too numerous to mention), that will in the end prevail. One day we will see the truth emerge and the scumbags (a mild description for these people), who are destroying our society, will be swept away.
Keep sticking it up 'em.
As Doug Keenan has lucidly pointed out in the past, if you don't have a model for your random process you can't make meaningful statements about things being "statistically significant" otherwise.
Yet a firmly entrenched part of Climate Science folk-wisdom is "it has to last thirty years or more to be climate" - otherwise it is random variation. Completely without any foundation so far as I can see - just a rule pulled from a hat.
Wriggle wriggle little worms
Hide decline and couch your terms
In sophist thought they spin and wheel
In place of truth we get 'look and feel'
I read your op-ed and think I understand a little bit if what the problem is — which for me (statistically speaking!) — is a great leap forward.
Thankyou for that.
As usual it's not the original error that is going to be somebody's downfall (I wish!) but the cover-up. But I fear you may have laid Mr McNeall's head on Auntie Julia's chopping block.
Backs against the wall of obfuscation. The extended version of the winter of our discontent. A wonderful essay Mr. Keenan. I wonder how many more times the question has to be asked? Maybe an FOIA question on the cost of not answering the question?
Anti-science rears its ugly head again. These 'scientists' obviously studied a different science to me. My tutor had a special file for results derived without showing any working; it is the same one that most of the Met Office's work should be going in by the look of it.
Whenever, the raw data, methodology or errors are missing I instantly get suspicious. Invariably, my suspicions are justified.
Hello all,
I'm pretty certain I'll have more to say on this later, but for the moment, please read this email I sent to Doug Keenan on 25th January 2013.
http://dougmcneall.wordpress.com/2013/04/09/some-more-correspondence-with-doug-keenan/
Thanks,
Doug McNeall
“Ministers should be as open as possible with Parliament, refusing to provide information only when disclosure would not be in the public interest”
Disclosure would absolutely not be in the public interest. The subsequent upheaval would be unthinkable. There would be riots.
This is important work. Very well done.
I've added this to my blog with the comment;
If we are in a situation where the UK’s premier weather research and forecasting organisation (Met Office) is being used as a tool to further the global warming meme, and will not justify its methodology to Parliamentary investigation, then we have to ask a simple question.
What is there to hide?
Interesting letter from Doug McNeall.
Interesting also in that it apparently contradicts the information previously provided by the Met Office to parliament:
The Answer claimed that “the temperature rise since about 1880 is statistically significant”.
Doug McNeall says:
In conclusion, my suggestion is that when asked if there has been a “significant” change in temperatures since the 1880s, we should say “yes”. If we are asked if there has been a statistically significant change in temperatures since the 1880s, we do not say “yes” or “no”, we say “that is not a valid question”.
Perhaps a better question would be :"has there been a material change of temperatures since the 1880's?
I suggest the answer would be no.
If the answer to your question is "the temperature rise since about XXXX is statistically significant” then you have asked the right question.
If the answer to your question is “the temperature rise since about XXXX is NOT statistically significant” then you have not asked a valid question.
Publicly funded institutes like the Met Office or the BBC have in the past paid little attention to the needs of the UK population, does anyone really think that is going to change unless their funding is taken away from them.
Doug McNeall says:
"I would suggest a Bayesian solution to this problem. I think the appropriate question is 'given that we see this these temperatures, what is the probability they are anthropogenically driven?' Using Bayes theorem, this combines the likelihood (from the first question), with the prior probability that global temperatures are anthropogenically driven, to some degree. Of course, this prior probability contains subjective judgements ..."
Bayesian methods cannot avoid begging the question.
Doug, perhaps you could also ask the Met Office what physics they are using to make the assumption (in their models) that CO2 has any effect on temperature whatsoever (and as always, refuse to accept any 'consensus' answer).
Doug McNeall at 9:34 AM
The problem is that you can not say (as the Met Service apparently said) it was "statistically significant" if a linear trend or even if an AR(1) process is assumed because the time series we have available violates the assumptions necessary to apply that test.
A couple of other niggles I have about your letter are:
First your characterisation of the problem at hand i.e. “given that we see this these temperatures, what is the probability they are anthropogenically driven?” I suspect that most would say the probably is high, we know CO2 has some effect. The real question is about the proportion that is anthropogenically driven. In this sense if we can demonstrate that the instrumental temp series prior to significant forcings fits a particular ARIMA model and the latter period isn't significantly different (i.e. a CO2 term is not significant in conjunction with that model) we have a basic result that is suggestive.
Second if our prior knowledge in a Bayesian sense is captured in CGMs (and there are technical reasons why this can't be the case) then we are getting close to rejecting the validity of those models too.
@Doug McNeall
thanks for coming over here and popping your head over the parapet :-)
With many on both sides of the debate looking to deploy statistics as a rigid pivot point for debate I don't envy your position...Some folk simply won't accept that they're not getting an answer because they've asked an unanswerable question.
however -
"A significance test attempts to answer the question “given that there was no anthropogenically
driven global warming, what is the probability that we would see these temperatures?” This is
interesting, but not really what we are looking for. As you and others have noted, this kind of test
does not allow you to distinguish between forced trends, and the degree of long term persistence
in the system. I would suggest a Bayesian solution to this problem. I think the appropriate
question is “given that we see this these temperatures, what is the probability they are
anthropogenically driven?” Using Bayes theorem, this combines the likelihood (from the first
question), with the prior probability that global temperatures are anthropogenically driven, to
some degree. Of course, this prior probability contains subjective judgements and information
from elsewhere, including fundamental physics."
- does that mean you make a guess, informed by subjective judgement?
Where is the devil's advocate in all this - and some hard estimates of the bounds of confidence?
I think Doug MacNeall makes an excellent argument in his correspondence posted above and in so doing he points up the very problem that is facing honest climate scientists and honest decision makers — who (I still believe) are in the majority.
Ah, but that's the thing, Doug. There are too many people — eco-activists, scientists and businessmen who know a milch cow when they see one, sh1t-stirrers (several names spring to mind), politicians who see a tax-raising opportunity — who are more than happy to "extrapolate a forecast of global temperature" and the scarier the better and, for better or worse, the Met Office has allowed itself to get tangled up in this farrago and apparently doesn't know how to get itself disentangled.I was particularly interested in (my emphasis)
I would suggest now might be a good time to take a deep breath, swallow hard, and come clean.
Doug McNeall says:
"I would suggest a Bayesian solution to this problem. I think the appropriate question is 'given that we see this these temperatures, what is the probability they are anthropogenically driven?' Using Bayes theorem, this combines the likelihood (from the first question), with the prior probability that global temperatures are anthropogenically driven, to some degree. Of course, this prior probability contains subjective judgements ..."
Bayesian solutions always beg the question.
Doug
This is great work and a fascinating story.
Of course the additional question can be asked as to how statstically significant the rise is compared to other eras?
I have been reconstructing CET (which many scientists believe ia a good proxy for Northern Hemisphere and Global temperatures) from its instrumental date of 1659 back as far as I can go. I am currently at 1538
http://climatereason.com/Graphs/Graph01.png
Research from many sources, including the excellent library and archives of the Met office itself, demonstrates that the fifty year period back from 1538 will be virtually indistinguishable from the modern era and slightly warmer than the rather alarming temperature decline we are currently witnessing in the UK
http://www.metoffice.gov.uk/hadobs/hadcet/
In talking of tenths of a degree we are dancing on the heads of pins. As Hubert Lamb once remarked. 'we can understand the tendancy (of temperatures) but not their precision.' His son Norman Lamb MP was, and probably still is, 'owning minister' at the met office. Perhaps he might be persuaded to take up the cudgels should you need answers to searching questions?
I look forward to the next instalment of this saga
tonyb
Doug Keenan:
Doug McNeall provided a number of numbers in his email. Unfortunately, they are all, without exception, dates.
Not exactly what was beeing looked for, I think ;(
It's really refreshing to see Doug here providing some clarification. By the time an internal report from a specialist has been interpreted by managers, fed to the chairperson / CEO's office and knocked about by speech writers...The tortured motivations of the folk at the head of government departments leads to some real foot meet mouth moments.
Sometimes though, their arrogance gets the better of them and quango-ista hubris trumps all. The poor old Met Office can't win sometimes. This 'ere miserable display must have toes curling at The Met Office and no mistake
I fail to see why parliamentary obstruction and misdirection is tolerated - add to that, the easiest deflection would be to use the web to kick the discussion off into a public forum somewhere where specialists can discuss the nuanced points of analysis at their leisure and swap data etc. etc. - too much to hope for and too subversive for the politicos and activists maybe?
"I refer the Honorable Lord to the statistical forum at www.metoffice.gov.uk where the issue is under analysis by people far better qualified than I to comment on the methodologies and results"
Fantasy, I know....
I read Doug's response, in so far as I have a grip on statistical analysis and modelling, he seems to be saying that the statistical significance can't be calculated so it's the wrong question. So, if I understand it correctly, 0.8C rise over the period 1880 - 2010 is significant, but it's statistical significance can't be calculated. He is probably correct, but I'd like to have the barmaid's explanation if I could.
Slightly O/T but Paul Homeward has an interesting post about a Met Off. briefing to the Environment Agency concerning causes of bad weather.
"not a lot of people know that"
tomo
I suppose the only flaw in the statistical forum idea is that no-one would be obliged to provide any answers, unlike a question in parliament.
Perhaps the nearest thing we have at the moment is My Climate and Me. Unfortunately, questions on this site are limited to 180 characters and not all questions are likely to receive an answer. But apart from that ...
Doug McNeall says:
"I would suggest a Bayesian solution to this problem. I think the appropriate question is 'given that we see this these temperatures, what is the probability they are anthropogenically driven?' Using Bayes theorem, this combines the likelihood (from the first question), with the prior probability that global temperatures are anthropogenically driven, to some degree. Of course, this prior probability contains subjective judgements ..."
Bayesian solutions are invariably agenda-driven.
The classical p-value has been getting a hard time on this site recently, and yet it is important for this discussion between the two Dougs. (Burns wrote a poem called 'The Twa Dugs' which naturally comes to the simple mind here. It is about two dogs in amiable conversation from either side of a considerable divide - in their case social, in ours possibly one of power: the Met Office has had power through influence, and the possibilities of more of that were presumably what attracted the climate zealot Robert Napier into getting in there at a high level. But I digress.)
I see the p-value as a measure of the strength of evidence in a data set, all by itself, for some supposition or other about change in the system which produced the data. In other words, we, for the time being, treat it in isolation from other things we may know. We need to agree that before this supposed change, the system was in State A, and we may well have grounds to suppose that the system was in a new state, State B, when the data was produced – this data being held to be to some extent different from the usual. It is being presented as evidence of change after all.
We forget about B for the moment. If the system had remained in State A, what is the probability that it could produce data (or some statistic derived from the data) at least ‘as unusual’ as the data we are looking at? If we can compute this, we have a p-value.
If the p-value is very small, we are more inclined to say the system was not in State A, but in some other state. We now turn to State B, our preferred alternative, with increased assurance that it is a better description of the new reality. We have not refuted A, nor proven B, but we have established that we have some strong evidence to hand that A may have changed.
If the p-value is large, we have not refuted B, nor proven A, we have merely found that the evidence in the data set is not at all convincing about the magnitude or direction of any change from A. In Scots Law, we would say ‘unproven’ would be our verdict based on that evidence alone – all the while we might still quite reasonably cling to the notion that B was a better choice, but hopefully admitting now that this particular piece of evidence, our data set, is weak and inconclusive. If we were detectives who saw that in advance, we would not have brought the case to trial yet. We would have gone in search of more evidence instead. In the climate context, we would not have sounded the alarum.
What are the problems here? There are many. First, how well can we define State A. Second can we compute some testable statistic based on this definition and, Doug K's key point, an associated statistical model? Third, how good is the data set we wish to use? Global mean temperature is an invention, a construct, and so we might well anticipate fun and games here. Ideally, for simple hypothesis testing, we’d like our data to be a random sample from the system we are agonising over. This oils the works of our inference process very nicely, but random samples are very hard to find! We have not taken random measurements in space or time of the climate system. We have generally taken them where they are convenient and/or important for us to do so. And of course, we have the problem of defining what is meant by 'large' and 'small' as we contemplate the p, or ideally, before we do so!
But Doug McNeal has taken a different tack, by going for the Bayesian option. All three problems still apply, but now we can step away from the isolated consideration of our data set, and consider it in the context of our prior knowledge, or, at least, of our prior beliefs. The evidence of witness D is to be heard only alongside a lot of context – perhaps the witness is disreputable, perhaps saintly. How we see the witness will determine how we treat his evidence. Thus if our prior belief is that the system has moved to State B, we fold in the data and lo and behold, it reinforces our belief. Our posterior belief is stronger than the prior. On the other hand, if our prior belief is that we are still in State A, we fold in the data, and lo and behold, we shall see a weakening (the data remember being somewhat ‘unusual’ for State A) but we do not necessarily need to discard our belief. Our posterior view may still be in favour of A. Only if the data set is dramatically different from ‘usual’, or our belief in A is very weak, will we be led to consider changing our minds.
The Two Dougs could be in for further debate!
Be that as it may, I presume we cannot yet take Doug MacNeal’s view as the official Met Office one? If not, then what is the official one?
I think Doug Keenan has done a tremendously useful job here. This is exactly the kind of examination of the evidence and of the associated reasoning processes that should have taken place as a matter of course – funded and audited by governments being presented with radical proposals affecting the well-being of citizens on an unprecedented scale. But they did not take place. Nullius in Verba was thrown out of the window not just by the short-sighted leadership of The Royal Society, but by government itself. Thankfully, we have had free citizens take up the challenge. Doug Keenan joins McIntyre and McKitrick on the statistical frontline. Heroes all.
Parliamentary questions are a good way of challenging the AGW premise.
The multiple meanings of 'significance' (Mr. McNeall identifies two of them) must not be allowed to cloud the issue. Statistical significance - of actual v observed - depends on the null hypothesis and departures-from. Parliamentarians should avoid referring to N-zero and drill down to the etymolgy behind it: "that which would otherwise have been the case".
The PQ should be: "In declaring a 0.8C rise in temperatures since 1880, what temperature does the government consider current and what would it otherwise have been had greenhouse gases not risen since then?" Put 'em on the spot.
The classical p-value has been getting a hard time on this site recently, and yet it is important for this discussion between the two Dougs.
(Burns wrote a poem called 'The Twa Dugs' which naturally comes to the simple mind here. It is about two dogs in amiable conversation from either side of a considerable divide - in their case social, in ours possibly one of power: the Met Office has had power through influence, and the possibilities of more of that were presumably what attracted the climate zealot Robert Napier into getting in there at a high level. But I digress.)
One can view the p-value as a measure of the strength of evidence in a data set, all by itself, for some supposition or other about change in the system which produced the data. In other words, we, for the time being, treat it in isolation from other things we may know. We need to agree that before this supposed change, the system was in State A, and we may well have grounds to suppose that the system was in a new state, State B, when the data was produced – this data being held to be to some extent different from the usual. It is being presented as evidence of change after all.
We forget about B for the moment. If the system had remained in State A, what is the probability that it could produce data (or some statistic derived from the data) at least ‘as unusual’ as the data we are looking at? If we can compute this, we have the p-value.
If the p-value is very small, we are more inclined to say the system was not in State A, but in some other state. We now turn to State B, our preferred alternative, with increased assurance that it is a better description of the new reality. We have not refuted A, nor proven B, but we have established that we have some strong evidence to hand that A may have changed.
If the p-value is large, we have not refuted B, nor proven A, we have merely found that the evidence in the data set is not at all convincing about the magnitude or direction of any change from A. In Scots Law, we would say ‘unproven’ would be our verdict based on that evidence alone – all the while we might still quite reasonably cling to the notion that B was a better choice, but hopefully admitting now that this particular piece of evidence, our data set, is weak and inconclusive. If we were detectives who saw that in advance, we would not have brought the case to trial yet. We would have gone in search of more evidence instead. In the climate context, we would not have sounded the alarum.
What are the problems here? There are many. First, how well can we define State A. Second can we compute some testable statistic based on this definition and an associated statistical model? Third, how good is the data set we wish to use? Global mean temperature is an invention, a construct, and so we might well anticipate fun and games here. Ideally, for simple hypothesis testing, we’d like our data to be a random sample from the system we are agonising over. This oils the works of our inference process very nicely, but random samples are very hard to find! We have not taken random measurements in space or time of the climate system. We have generally taken them where they are convenient and/or important for us to do so.
But Doug McNeal has taken a different tack, by going for the Bayesian option. All three problems still apply, but now we can step away from the isolated consideration of our data set, and consider it in the context of our prior knowledge, or, at least, of our prior beliefs. The evidence of witness D is to be heard only alongside a lot of context – perhaps the witness is disreputable, perhaps saintly. How we see the witness will determine how we treat his evidence. Thus if our prior belief is that the system has moved to State B, we fold in the data and lo and behold, it reinforces our belief. Our posterior belief is stronger than the prior. On the other hand, if our prior belief is that we are still in State A, we fold in the data, and lo and behold, we shall see a weakening (remember, the data seems somewhat ‘unusual’ for State A) but we do not necessarily need to discard our belief. Our posterior view may still be in favour of A. Only if the data set is dramatically different from ‘usual’, or our belief in A is very weak, will we be led to consider changing our minds.
The Two Dougs could be in for further debate!
Be that as it may, I presume we cannot yet take Doug MacNeal’s view as the official Met Office one? If not, then what is the official one?
I think Doug Keenan has done a tremendously useful job here. This is exactly the kind of examination of the evidence and of the associated reasoning processes that should have taken place as a matter of course – funded and audited by governments being presented with radical proposals affecting the well-being of citizens on an unprecedented scale. But they did not take place. Nullius in Verba was been thrown out of the window not just by the short-sighted leadership of The Royal Society, but by government itself. Thankfully, we have had free citizens take up the challenge. Keenan joins McIntyre and McKitrick on the statistical frontline. Heroes all.
Am I right in thinking that there appears to be a genuine statistical disagreement here between DK and DM ?
DK seems to think one can meaningfully answer "yes" or "no" to the question: for a given statistical model, is the warming of 0.8C since 1880 statistically significant, whereas DM seems to thinks not.
Maybe slightly o/t, but still the Met Office.
Most days I visit their site in order to compare forecast with reality, and did so earlier after reading this post.
For a long time their banner headline has read "Weather and Climate Change", indeed my 'bookmark' remains as such. Today, that headline has disappeared and become simply "Met Office".
Odd?
Yes - well spotted, mikemUK.
Google still shows the link as Met Office: Weather and Climate Change .
If you put "climate change" into the Met Office search box you get:
Climate change is a complex subject, with genuine areas of uncertainty and scientific controversy. On this page you will find all the key facts about climate change.
Not sure whether this has been recently updated, but it looks like the science is looking less settled. I wonder if this is a consensus view being put forward by the Met Office?
matthu
ah ... yes... but ...
The formal nature of the Q&A in the parliamentary arena means that generally, extended explanations cannot be given - and I don't mean Filibustering style explanations - it's just that some answers are necessarily long and when the majority of listeners in the chamber haven't a clue what it's about and time is at a premium - it's very (far too) easy for the respondent to rapidly slime off awkward questions - a tactic I've witnessed more times that I care to face up to and in my view one of the most deeply troubling aspects of both chambers.
In my view most parliamentary questions are simply "declarations of interest". The Met Office most definitely has the human resources to operate a bit of meaningful engagement but the paradigm they deploy now is the 20th century broadcast model rather than 21st century crowd sourcing. Yes - you are going to get pedantic, deluded wingnuts trying to shout people down but decorum and reasonableness generally win out. You can of course ban people as a last resort - or even better imho, deploy a public compost heap of spiked posts (and IP addresses Z). If The Met Office want to be perceived as NOT another self regarding dysfunctional bureaucratic quango - they'd better start upping their game and no mistake.
Yes, a parliamentary question is a form of ritualised confrontation - but there's also a ritualised rat hole mechanism...
mikemUK, I still see the page title as "Weather and Climate Change - Met Office".
" If we are asked if there has been a
statistically significant change in temperatures since the 1880s, we do not say “yes” or “no”, we
say “that is not a valid question”. "
Surely Doug McNeall has this the wrong way round. The question is a perfectly valid one, the answer rather should be; "There is no valid answer".
I think Doug Keenan has done a tremendously useful job here. This is exactly the kind of examination of the evidence and of the associated reasoning processes that should have taken place as a matter of course – funded and audited by governments being presented with radical proposals affecting the well-being of citizens on an unprecedented scale. But they did not take place. Nullius in Verba was been thrown out of the window not just by the short-sighted leadership of The Royal Society, but by government itself. Thankfully, we have had free citizens take up the challenge. Keenan joins McIntyre and McKitrick on the statistical frontline. Heroes all.
PS This is the last paragraph of a far-too-long comment sent earlier, probably twice in my puzzlement. Are comments above a certain length automatically deleted by the software? Or, heaven forfend, were the contents rejected?
Whoops.
What happens if we get 10 heads in a row, the statistics tells us the coin is rigged, but we know it is not. Do we go with reality or with statistics?
From the article by McNeal, about a Bayesian approach to answer the question "given that we see this these temperatures, what is the probability they are anthropogenically driven?”:
"Of course, this prior probability contains subjective judgements and information from elsewhere, including fundamental physics."
This seems to me circular reasoning in support of a specific political ideology.
'....Ministers who knowingly mislead Parliament will be expected to offer their resignation to the Prime Minister..'
Oh...
Oh...
I think we could name a few, couldn't we..?
If the question cannot be answered by a simple yes or no, then any answer must be a matter of opinion.
Who is to say, in that event that one opinion is more valid than another.
@ Martin A
a firmly entrenched part of Climate Science folk-wisdom is "it has to last thirty years or more to be climate" - otherwise it is random variation
I think it likelier that, as they've only got 150 years of data, they had to divide that into chunks small enough to support the purported observation of supposed trends.
At the same time, though, the purported "cycle" needed to be long enough that they'll be safely in receipt of their final salary pensions before being exposed as mediocre charlatans.
30 years meets both criteria: it gives you five periods of weather to study and a whole career in which to do so, immune from being found out.
J4R: "30 years meets both criteria: it gives you five periods of weather to study and a whole career in which to do so, immune from being found out."
I wouldn't want to be one of Hansen's grandchildren...
... According to the Companion to the Standing Orders and Guide to the Proceedings of the House of Lords, §4.68 Ministerial Responsibility, “Ministers should be as open as possible with Parliament, refusing to provide information only when disclosure would not be in the public interest” ...
"Minister, it is obviously in the public interest to prevent the Met Office from looking like a fool. After all, you have authorised the expenditure of £33m for a new computer system which provides these figures - you continue to fund them to the tune of £200m per year, even though their forecasts are bywords for inaccuracy. If you now let them admit that they have no meteorological competence, you will give the Opposition a field day, and will be hammered in the House yourself for misspending public money.
Surely the most statesmanlike action is to draw a veil over the proceedings? After all, if this revelation proves to be the start of a general unraveling of the climate change policy, I cannot see that the Prime Minister will smile upon you. His father in law has extensive interests in that field. As, indeed, have many of your Cabinet colleagues and other establishment figures, including the entire BBC pension fund. Would it be wise to make enemies of all these people at this stage of your career?"
(Note from Sir Humphrey to the Minister)
The impact of human-generated CO2 on temperatures is generally said to have begun after ~ 1950. If so, would it be reasonable for the Lord to ask something like the following question:
"Is there a statistically significant difference in the changes in temperature from 1950 to the present, and from 1880 to 1950?"
Can statistics even be applied to this question?
DL. I asked Doug something similar on his blog, it was to the effect that if asked would he say the temperature rise of approx 0.4C between 1880 and 1940 was significant would he say yes?
Wanna hear something funny?
Just watching cnn weather here in the uk.
Rising co2 causes greater turbulence and is a danger to flights.
I quote "Better hang on tight!"
Now stop laughing! get up off the floor, and dry your eyes,you chaps have work to do:)
I mentioned this at wattsup,got to share the laffs!
Now wheres Josh?:)