Nov 24, 2009

The CRU emails

Since the release of the "hacked" (or leaked) emails from the server of the CRU at the University of East Anglia (UK), there has been a lot of commentary, argument and opinions expressed regarding the content and implications of the emails, appearing on blogs and in the news media. 

The opinions can roughly be divided as follows:
1) The emails don't really mean anything, they are simply private correspondence containing "robust" discussions, taken "out of context" (for some reason, "robust" seems to be a favourite word)
see e.g.
http://www.realclimate.org/index.php/archives/2009/11/the-cru-hack/
http://www.latimes.com/news/nation-and-world/la-fg-climate-hacker22-2009nov22,0,913036.story
http://www.guardian.co.uk/environment/cif-green/2009/nov/23/leaked-email-climate-change
http://blogs.news.com.au/dailytelegraph/yoursay/index.php/dailytelegraph/comments/you_cant_immunise_against_arrogance/

2) The emails are concerning and reflect badly on the main parties involved, but may not really affect general climate science very much.
see e.g.
http://www.monbiot.com/archives/2009/11/23/the-knights-carbonic/
http://www.news.com.au/story/0,1,26386792-401,00.html

3) The emails expose deeply disturbing practices at the very top of climate science which seriously undermine the integrity of those involved.  The emails also taint the related scientific conclusions, though the emails do not themselves prove that global warming is false.
see e.g.
http://online.wsj.com/article/SB10001424052748704888404574547730924988354.html?mod=WSJ_hpp_sections_opinion
http://www.timesonline.co.uk/tol/comment/columnists/guest_contributors/article6927598.ece
http://blogs.telegraph.co.uk/news/jamesdelingpole/100017546/climategate-why-it-matters/

4) The emails prove a giant global conspiracy to lie to the public about global warming.  The perpetrators know that warming is a lie and are simply perpetuating this lie for profits.
(see comments on various blogs - I can't actually find a blogger or columnist/reporter who seems to actually believes this)

I would say I fall into category three above.  Before going further, I want to share an excerpt from an excellent speech made by Richard Feynman, winner of the Nobel Prize for Physics, regarding the scientific method:

Now it behooves me, of course, to tell you what [is] missing [in bad science].
That is the idea that we all hope you have learned in studying science in school--we never explicitly say what this is, but just hope that you catch on by all the examples of scientific investigation. It is interesting, therefore, to bring it out now and speak of it explicitly. It's a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty--a kind of leaning over backwards. For example, if you're doing an experiment, you should report everything that you think might make it invalid--not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you've eliminated by some other experiment, and how they worked--to make sure the other fellow can tell they have been eliminated.

Details that could throw doubt on your interpretation must be given, if you know them. You must do the best you can--if you know anything at all wrong, or possibly wrong--to explain it. If you make a theory, for example, and advertise it, or put it out, then you must also put down all the facts that disagree with it, as well as those that agree with it. There is also a more subtle problem. When you have put a lot of ideas together to make an elaborate theory, you want to make sure, when explaining what it fits, that those things it fits are not just the things that gave you the idea for the theory; but that the finished theory makes something else come out right, in addition.

In summary, the idea is to try to give all of the information to help others to judge the value of your contribution; not just the information that leads to judgment in one particular direction or another.

The easiest way to explain this idea is to contrast it, for example, with advertising. Last night I heard that Wesson oil doesn't soak through food. Well, that's true. It's not dishonest; but the thing I'm talking about is not just a matter of not being dishonest, it's a matter of scientific integrity, which is another level. The fact that should be added to that advertising statement is that no oils soak through food, if operated at a certain temperature. If operated at another temperature, they all will-- including Wesson oil. So it's the implication which has been conveyed, not the fact, which is true, and the difference is what we have to deal with.

We have learned a lot from experience about how to handle some of the ways we fool ourselves. One example: Millikan measured the charge on an electron by an experiment with falling oil drops, and got an answer which we now know not to be quite right. It's a little bit off, because he had the incorrect value for the viscosity of air. It's interesting to look at the history of measurements of the charge of the electron, after Millikan. If you plot them as a function of time, you find that one is a little bigger than Millikan's, and the next one's a little bit bigger than that, and the next one's a little bit bigger than that, until finally they settle down to a number which is higher.

Why didn't they discover that the new number was higher right away? It's a thing that scientists are ashamed of--this history--because it's apparent that people did things like this: When they got a number that was too high above Millikan's, they thought something must be wrong--and they would look for and find a reason why something might be wrong. When they got a number closer to Millikan's value they didn't look so hard. And so they eliminated the numbers that were too far off, and did other things like that.
After reading those excellent words from Feynman, I then turn to some less excellent words found in these emails:
Mike,

Can you delete any emails you may have had with Keith re AR4?
Keith will do likewise. He's not in at the moment - minor family crisis.
Can you also email Gene and get him to do the same? I don't have his new email address.
We will be getting Caspar to do likewise.
I see that CA claim they discovered the 1945 problem in the Nature paper!!
Cheers
Phil
Note: "AR4" means the latest IPCC report, the 4th Assessment Report
The "Mike", "Keith" and "Phil" were all major contributors
to the 4th IPCC report (2007).
Mike,
I presume congratulations are in order - so congrats etc !
Just sent loads of station data to Scott. Make sure he documents everything better this time! And don't leave stuff lying around on ftp sites - you never know who is trawling them. The two MMs have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I'll delete the file rather than send to anyone. Does your similar act in the US force you to respond to enquiries within 20 days? - our does ! The UK works on precedents, so the first request will test it. We also have a data protection act, which I will hide behind. Tom Wigley has sent me a worried email when he heard about it - thought people could ask him for his model code. He has retired officially from UEA so he can hide behind that. IPR should be relevant here, but I can see me getting into an argument with someone at UEA who'll say we must adhere to it !
Note: "CRU station data" refers to the raw temperature
data collected by the Climate Research Institute and used
to calculate global average temperatures.
"MM" refers to Steve McIntyre and Ross McKitrick.

Phil and I have recently submitted a paper using about a dozen NH records that fit this category, and many of which are available nearly 2K back--I think that trying to adopt a timeframe of 2K, rather than the usual 1K, addresses a good earlier point that Peck made w/ regard to the memo, that it would be nice to try to "contain" the putative "MWP", even if we don't yet have a hemispheric mean reconstruction available that far back [Phil and I have one in review--not sure it is kosher to show that yet though--I've put in an inquiry to Judy Jacobs at AGU about this].

Note: MWP means "Medieval Warm Period".
Thanks, Phil.

The stuff on the website is awful. I'm really sorry you have to deal with that kind of crap. If the RMS is going to require authors to make ALL data available - raw data PLUS results from all intermediate calculations - I will not submit any further papers to RMS journals.


Cheers,
Ben
Note: RMS = Royal Meterological Society, which publishes the
"International Journal of Climatology", "Atmospheric Science
Letters", and "Weather", and other journals.
Tim,
Attached are the calibration residual series for experiments based on available networks back to:
AD 1000
AD 1400
AD 1600
I can't find the one for the network back to 1820! But basically, you'll see that the residuals are pretty red for the first 2 cases, and then not significantly red for the 3rd case--its even a bit better for the AD 1700 and 1820 cases, but I can't seem to dig them up. In any case, the incremental changes are modest after 1600--its pretty clear that key predictors drop out before AD 1600, hence the redness of the residuals, and the notably larger uncertainties farther back...
You only want to look at the first column (year) and second column (residual) of the files.
I can't even remember what the other columns are!
Let me know if that helps. Thanks,
mike
p.s. I know I probably don't need to mention this, but just to insure absolutely clarify on this, I'm providing these for your own personal use, since you're a trusted colleague. So please don't pass this along to others without checking w/ me first. This is the sort of "dirty laundry" one doesn't want to fall into the hands of those who might potentially try to distort things...

There are many, many more troubling emails.  Regarding my preference for category three above I would say this.  First, these emails contain more than just robust discussion.  Discussing the deletion or withholding of scientific information is not robust, it is troubling.  As for the claim that statements have been taken out of context - the beauty of this situation is that anyone can work out the context if they read the emails in full and put some effort into doing some basic background research on the points they are unfamiliar with.  For me, these considerations rule out category one.  Category two is ruled out because if some of the key scientists are of questionable integrity, all their work is now suspect.  Not wrong, but suspect.  And all their public statements about consensus, about there being "no doubt" are suspect.  So I find myself agreeing with category three.  Global warming has not been disproved nor proved a hoax.  But the integrity of those at the top is now open to question - and thus, so is the quality of their work.

For more info, see:
http://blogs.sciencemag.org/scienceinsider/2009/11/in-climate-hack.html - good article
http://www.eastangliaemails.com/index.php - searchable full-text of all emails
http://bishophill.squarespace.com/ - blog detailing some of the problems.
http://camirror.wordpress.com/ - run by Steve McIntyre, a key sceptic mentioned in many of the emails.

{Addendum - the emails are not the whole story regarding the released data.  Regarding some of the other files, see: http://bishophill.squarespace.com/blog/2009/11/23/the-code.html for a good, regularly updated summary).

No comments:

Post a Comment