This about sums it up

From the hand of an unknown programmer trying to implement the Hadley Centre CRU data which Phil Jones and his rapscallion crew of artful dodgers tried to hide from us despite it being produced on our tax money:

OH FUCK THIS. It's Sunday evening, I've worked all weekend, and just when I thought it was done I'm hitting yet another problem that's based on the hopeless state of our databases. There is no uniform
data integrity, it's just a catalogue of issues that continues to grow as they're found. [Emphasis added]

Thanks to WUWT.

Comments

[dearieme]

They are not really databases, more a datamidden.

[David Duff]

Heh,heh, heh, nice one, 'DM'.

[hank]

David

I used to be a progrqmer. The Emails are only 5% of the dump. To me the programs are the damming thing.

http://pajamasmedia.com/blog/climategate-computer-codes-are-the-real-story/ Climategate Computer Codes Are the Real Story By Charli Martin at Pajamas Media.

Read the whole thing. Don't worry about the computer words read the frustration in person who constructed the HARRY_READ_ME file. Without knowing anything about programing you can also see the intent. The nice thing about this sort of file is that it is the programers notes to self. Normally, such a file will never be read by anyone other than the programer or the next programer, Thus it gets no higher lever review for "corretness".

The concluding comments.

I think there’s a good reason the CRU didn’t want to give their data to people trying to replicate their work.
It’s in such a mess that they can’t replicate their own results.

This is not, sadly, all that unusual. Simply put, scientists aren’t software engineers. They don’t keep their code in nice packages and they tend to use whatever language they’re comfortable with. Even if they were taught to keep good research notes in the past, it’s not unusual for things to get sloppy later. But put this in the context of what else we know from the CRU data dump:

1. They didn’t want to release their data or code, and they particularly weren’t interested in releasing any intermediate steps that would help someone else

2. They clearly have some history of massaging the data — hell, practically water-boarding the data — to get it to fit their other results. Results they can no longer even replicate on their own systems.

3. They had successfully managed to restrict peer review to what we might call the “RealClimate clique” — the small group of true believers they knew could be trusted to say the right things.
As a result, it looks like they found themselves trapped. They had the big research organizations, the big grants — and when they found themselves challenged, they discovered they’d built their conclusions on fine beach sand.

But the tide was coming in.

[David Duff]

Yes, Hank, others have pointed out that the codes are where the bodies are buried. I have read your link's name on other sites and I must get round to him - but there's just so much of the damned stuff to read!

By the way, I hope you and yours are enjoying a super Thanksgiving.