Doug Keenan has followed up on his observations about the long-term rainfall records for England and Wales with an exchange of emails with Julia Slingo, the chief scientist at the Met Office. (Note that images can all be enlarged by clicking on them).
Dear Julia,
On November 12th, I sent an e-mail, in which you were Cc’d, about the statistical analysis of (observational) climatic data that has been done by the Met Office. My e-mail stated that some of the analysis is so incompetent that it “is not science”. It then asked if scientists at the Met Office had training in the relevant branch of statistics—i.e. in time series.
You did not reply to that e-mail. In consequence, on November 29th, Lord Donoughue put the following Question in the House of Lords.
To ask Her Majesty's Government how many scientists employed by the Met Office spend at least half of their working time conducting work closely related to global warming; how many of those scientists have taken an undergraduate-level course in statistical analysis of time series; and how many of those scientists have taken a graduate-level course in the statistical analysis of time series.[HL3705]
The Answer does not seem to have been published, but Lord Donoughue has told me that the Met Office claims that it has tens of scientists who have at least some training in the statistical analysis of time series. Specifically, the Answer claims that there are tens of scientists who understand that modeling autoregression is essential in such analysis. That is very good to learn. I find it difficult to believe, though, given the analyses previously produced by the Met Office.
Yesterday, the Met Office issued a news release about 2012 weather statistics:
http://www.metoffice.gov.uk/news/releases/archive/2013/2012-weather-statistics
The news release indicated that rainfall is increasing. It gained considerable attention, e.g. at the BBC and Channel 4.
The news release is largely based on a statistical analysis of rainfall data. Few details are given, but it is obvious that the analysis did not properly model autoregression. When proper modeling is done, I am confident that there will not be a significant increase in either rainfall or extreme rainfall events. Indeed that would be obvious to anyone knowledgeable in time series.
I ask that the Met Office address the statistical issue responsibly—specifically, via the following.
1. Promptly issue a news release stating that the statistical analyses that underpinned yesterday’s claims about increasing rainfall were invalid.
2. Implement procedures to ensure that news releases based on statistical analyses are vetted by someone competent in the relevant statistics.
Sincerely, Doug
Slingo's response was as follows:
Dear Doug,
Thank you for your email.
Yesterday’s announcements report on the UK’s observed rainfall records. What was announced was that provisional data show that 2012 was the second wettest year in the UK national record, along with preliminary research on the frequency of 1 in 100 day rainfall events over the last 50 years. A few points in particular stand out:
- 2012 is one of the wettest years on record (since 1910).
- Two months last year (April and June) were the wettest in a series back to 1766.
- Annual rainfall totals for UK are higher in recent years than during the 20th century.
- 4 of the 5 wettest years have been since 2000
- We have observed more wet days (>99th percentile) in recent years than a few decades ago.
These are basic conclusions from the observational record and make no attempt at defining trends, or attributing causes, not least as the observational record of daily rainfall records is not sufficiently long to derive robust statistics.
The widespread coverage of the observed data reflect the real impact that the rain has had on people and the nation’s infrastructure in recent years.
Regards,
Julia
Keenan has now responded in turn:
Dear Julia,
Your message cites “a series back to 1766”. I did not previously realize that the Met Office had precipitation data going back that far. Having such data is highly valuable, of course, and much facilitates analysis.
The cited series is not for the whole UK, but rather for England & Wales. That is fine though, especially as nine out of ten UK residents live in England and Wales—so this is what most people will be concerned about. I appreciate that data for the earlier years might be somewhat inaccurate. Using the annual data, though, should help to smooth out problems (i.e. the errors probably partially cancel each other out). Thus, this all seems to be fair. A plot of the annual data is below.
From the plot, it appears that we should not have concern about recent annual precipitation being overly high. Moreover, during the last decade, 2011 (787 mm) and 2003 (761 mm) are particularly low.
By omitting the above plot, and any mention of the information in the plot, the Met Office would seem to have misled the public and the media. Consider this: if a listed company reported on its financial position, and omitted data that was as material to its position as the above plot is to the state of precipitation, the people who were responsible would probably face prosecution.
The precipitation data seems to be a somewhat like (fractional) Brownian motion, which is a type of random process. Some plots that were generated using Brownian motion are appended, for comparison. I am not claiming that precipitation actually is Brownian, merely suggesting that if the Met Office wanted to do research on the precipitation data, that might be a place to start.
Your message also cites some more recent observational data and indicates that such data supports the suggestion of significant increases. That is not possible, even in principle. In statistical analysis, inferences are not drawn directly from data. Rather, a statistical model is fit to the data, and inferences are drawn from the model. We sometimes see statements such as “the data are significantly increasing”, but this is loose phrasing. Strictly, data cannot be significantly increasing, only the trend (etc.) in a statistical model can be.
If we use a Brownian process as the model for the annual data, then there is no significant increase. Perhaps some non-Brownian models would be better, however, and would support an increase. If so, then it is the responsibility of the Met Office to select models to substantiate that. The Met Office, however, apparently has nothing to substantiate that. It has not even selected any model. And the same is true for the daily data.
It is because the Met Office has repeatedly had such serious failures in its statistical analyses that Lord Donoughue put his question in the House of Lords. In answer to the question, the Met Office apparently assured Lord Donoughue that it does have the requisite statistical expertise. Yet now, it issues a news release based on work that is so incompetent that if the work were submitted in an introductory (undergraduate) course, the student would flunk.
The answer to the question put by Lord Donoughue specifically acknowledged that modeling autoregression is essential. Did the “preliminary” research by the Met Office consider autoregression? No. Is it plausible that research that does consider autoregression will support an increase in precipitation? No. Was the Met Office aware that there is no valid evidence for an increase when it issued its news release? Either it knew, and thus acted fraudulently in issuing the release, or it was misleading when it told Lord Donoughue that it has the requisite expertise. (I suspect the latter.)
Taxpayers pay for the Met Office to provide expertise with climatic analysis. They are not getting anything close to what they pay for.
Sincerely, Doug
Sample plots generated via Brownian motion (the variance and dimension are as per the precipitation data):
;)
;)
;)
;)