Studying Temperature Data Using the Language of Science

Which of course, given the fact of the history of science....a religion founded upon tentativeness and assumed certainty..Yep, that's a religion alright.

 

PeakProphet wrote: “What is worth considering is that someone thought this was a good idea of temperature data collection. And this single example calls into question the judgement of whoever employed such a person, authorized the money to create such temperature station, the intellectual capability to even understand data collection, and whether or not this pollution of basic process propagated throughout the entire climate science world. 3rd party independent review matters and exists for a reason, and the very idea that some group or niche in science would avoid it, while ALLOWING examples like this to occur, is itself a telling concept.”

The satellite image led me to a graph at a hoax site contending that when U.S. airport-based data is excluded, the avg daily temp is lower. Accordingly, this is particularly so as of late. The chart may be unsourced, the product of hoaxers. I think so because it can’t be true that over the last hundred years or so the U.S. avg daily temp would be anywhere from 10 to 14 degrees F, as the chart indicates. They must mean degrees C. They say the data is from the GHCN daily temperature record. GHCN is a foundation, not a temperature data set in of itself. They provide raw data, but after adjustment processes, there are multiple reconstructions, some are known to have a cool bias. Below is the url to the argument & graph.

https://stevengoddard.wordpress.com...us-ghcn-stations-indicates-uhi-contamination/

I suspect the argument is flawed or misleading in some other manner. There’s a whole land-based temperature reconstruction community out there that I would recommend you explore. Perhaps you could find the correlation matrix you speak of? While these two sites are pro-AGW, that doesn’t mean they can’t assist you in finding the data you’re looking for.

If my understanding is correct, a step-change is a sudden, non-continuous swing, not unlike jet plane exhaust or perhaps even relying on data from a location that would initially show a warming. But apart from all of this, I think that if using data from stations at airports produced a bias, it couldn’t continue showing a false increase with subsequent readings.

 

Might be a good idea at some point. Would probably like to do a bit more EDA before trying to wade through all the other approaches people have taken to trying to pin down some of the fundamentals.

Hobbyists aren't where it should reside. The idea that you think they should have it is quite surprising, considering it is fundamental to assumptions made when two or more temperature stations are involved, let alone the entire grid system that climate folks have built out.

Good data collection is fundamental to doing good science. It might be completely possible to speculate that only within the past several decades have the issues of land and sea based temperature measurement bias, errors, bad sighting and whatever else been sorted through well enough to generate consistent temperature measurements on a large enough scale to be relevant. Bet I could make a case for that based on TOBS corrections, let alone these kinds of sighting shenanigans or differences between thermometer types and undoubtedly several other factors.

 

Science is founded upon the assumption that we cannot know everything (not tentativeness) and quantifies (not assumes) that certainty. If that is you're definition of a religion, then I am no longer an atheist.

 

As evidenced by the Met Office and numerous published papers, average temperatures have been calculated using the same methods for decades, knowing full well the potential for bias and errors in these measurements that might influence the results. Don't you think if there was some fundamental statistical flaw in their methodology, someone would have written about it by now?

 

A good read.

Understanding adjustments to temperature data

 

Sure. And that is why the TOBS correction for the Idaho station in question starts in1985. Because between 1914 and 1985 they knew everything they needed to know.

Apparently when it came to Mann's work, someone did. But more importantly, your question implies that because they can't be bothered with the same kind of 3rd party independent review that other scientists have, it must be because they know their work is wonderful?

How about we reverse that thought…because it can be so easily proven that assumptions of normality aren't holding up in even a single stations TMax temperature data, all temperature aggregations are invalid because they are based on such a false assumption?

 

Indeed.

 

Which prevents us from calculating average temperatures how? Apparently you don't think we can ever know anything because we will always be working from incomplete data.

Nobody is stopping any 3rd party independent review of climate data. Isn't that exactly what Berkeley Earth was? Maybe the ASA should host a workshop where leading statisticians can see how climate scientists are handling their data and see if they are reaching reasonable conclusions. Oh wait, they already have.

I suggest you write a paper explaining how all temperature aggregations are invalid because TMax temperature data does not fit assumed normal distribution and submit it to the JASA for publication. Let us know what they say.

 

We can calculate means all day long. All sorts of which ways, after all sorts of corrections, adjustments and whatnot. It appears to be a favorite assumption, that the mean best represents temperature.

You do not understand basic uncertainty, nor my experience with it. There is a saying at the office we have, when dealing with systems that are missing all sorts of data, and it is "unencumbered by data, we forge ahead". Compared to my world, this temperature stuff is a walk in the park.

No one is apparently asking either. Whereas the science I am familiar with, it REQUIRED a third party review. Maybe some science fields are just more careful then others?

Not as far as I could tell. Are you saying that Berkeley Earth a sub-committee of the ASA, tasked with making sure the statistical assemblies, aggregations, and correlations, with the full cooperation of whoever does the "official" methods and records keeping of climate change science?

Workshops are great! But I'm looking for the verification of the fundamentals, would you happen to have a link to the source material utilized during such reviews, the correlations and statistical sensitivity, tables within the Appendix perhaps showing the percentage of tested distributions that meet some requirement of normality, the number of stations not showing the claimed trend, you mean that sort of stuff?

That I would be fascinated with, as it pertains to the proper manipulation of data, as compared to just, how did you put it, "can see how". I've been showing you and everyone else what I've been putting together so you "can see how" and yet I can't get a single person to refute not a single number, distribution, range or statistical test yet.

I've got 3 sitting on my desk right now, papers held to the standards of science I am familiar with. You are seeing nothing here but some basic EDA, just go find yourself a textbook on the topic and begin reading if you are confused about it. Just the idea that you think basic EDA is worthy of a peer reviewed science paper is...amusing.

 

True, but if it does not conform to the official government review, those people that do are called deniers.

 

Not only is the assumption that the mean best represents temperature useful in climate science, it appears to be very useful for teaching introductory statistics. It would be really embarrassing if they were teaching bad statistical methods, wouldn't it?

http://www.amstat.org/publications/jse/v21n1/witt.pdf

If I remember correctly, what you do is engineering, not science. Care to show actual science field that requires third party review?

I'm saying that Berkeley Earth is a third party that among other things, looked at the statistical methodology used to calculate mean temperatures and failed to find a problem.

Being as familiar with statistics as you claim to be, I would think even you could navigate the ASA website and find this information. Here are a couple of links that should help you get started.

http://www.image.ucar.edu/Workshops/2007/ASAclimate/index.shtml
http://www.amstat.org/committees/ccpac/

It doesn't need to be peer reviewed, just a comment paper would be sufficient. If basic EDA is all that you need to disprove decades of climate science, I think they would be eager to publish. With several other articles dealing with climate, temperature and uncertainty, it's not like they're avoiding the subject.

 

Absolutely. But the manipulations, interpolations, aggregations, correlations and extrapolations utilized by climate folks is in no way introductory.

I explain my experience on the first page. And of course I have both been subjected to 3rd party review, and participated in the review of the work of others, although not for the ASA, with in conjunction with some of its members.

I understand what you said. I asked for a reference to the particulars.

Cool! A workshop!! I'm certainly glad that the professionals are being consulted….now can you point me to the ringing endorsement of..say…a correlation matrix like other scientists use to weave together at LEAST the temperature stations in…say…a single state?

You base this opinion on how many years as a scientist, and how many publications to your name?

And you base this opinion on many years as a scientist, and how many publications to your name?

And you need to read better, I haven't said a word about DISPROVING anything. I am examining data, which is what scientists do prior to forming their assumptions, confidence intervals. I then compare the examination of that data with the assumptions that others have made, and along the way ask some obvious questions. Why would anyone ever even be unhappy with that, it is how science is done, nothing more. Certainly I will never sacrifice honesty to create scary scenarios, because that isn't the science I was taught. I can't speak for others.

 

And yet none of the ASA experts who have met with the climate scientists behind the manipulations, interpolations, aggregations, correlation and extrapolations have expressed any concerns about the process.

Doesn't answer the question. Which actual science field requires third party review?

The paper I linked to was the particulars.

Maybe you should ask the professionals associated with the ASA. I'm sure they could explain why it's irrelevant much better than I could.

Absolutely none. But I do have ample experience weighing conflicting arguments and determining which one is failing to support its position.

I'm sorry, I was under the impression this whole thread was intended to disprove that anthropogenic greenhouse gas emissions "are extremely likely to have been the dominant cause of the observed warming since the mid-20th century."

 

Aberdeen Station Mean Temperatures of Daily Average Temperatures

People have been mentioning lately that they aren't interested in the fine levels of detail available in the climate world, but wanted a yearly mean because that is the metric they are comfortable with. So I thought I would whip out one of those for folks.

This is a graph not of TMax now, but of TAvg calculated without TOBS corrections from 1914 to 2014. Ignore 1914 (the first data point on the far left) as it is only a partial year, but I didn't wish to remove it from the dataset. Turns out, the TOBS correction did some REALLY weird things to the data and until I can figure out if I calculated it correctly or not, or misapplied it in some way, I'll stick with the traditional approach of honoring data and not getting into any "correction" arguments yet.



Of interest, science of language speaking, we are now dealing with multiple levels of averaging...first we average the day...and then we average all the days to a yearly level. This immediately begs the question of assumed normality yet again, because in statistics we should now be able to assume normality.

Out of 101 parameterized fits, only 8 were best fit with a normal distribution.

Here is a histogram of those 101 yearly means. Notice that this represents uncertainty around an ANNUAL MEAN now, in other words a yearly mean at this station can vary across a range of 3.36C for an 90% confidence interval. Not fractions of 1F, or fractions of 1C. Call it +/- 1.65C, roughly.



I also provided 4 different fits to the data, but under none of the previously mentioned statistical tests was a normal distribution the best fit. 3 of the 4 were symmetrical however, and this is actually a good sign for any assumptions anchored on the central limit theorem.

 

Idaho Mean Yearly Temperature

So now I've got about 100MB of data from multiple Idaho stations to begin testing assumptions of correlation. But just for fun I thought I would cough up this little graph. From these 25+ temperature stations in Idaho with records going back across the past century, mean by year of calculated dailyT_Avg using the NOAA quality controlled temperature data set, we now have more spatial control over how global climate change has changed the temperature in Idaho.

 

can't see the graph.

 

Damn. Okay, how about this one?

I think it is quite interesting how the effects of global warming have spilled over into Idaho.

 

got that one

Nice, one hundred years and the temperature didn't move. I believe this is most likely the case for most of the Northern Hemisphere. But I have no records, but the readings I have looked at around the globe don't show an increase in time as each differed from the years the stations were active. IMO, the biggest complainers of the heat are from Australia. In fact, on here and other boards, the Aussie rule seems to be invoked frequently.

Thanks,

 

Now let's be fair. It hasn't moved...YET. Obviously the models that have been predicting temperature change for decades now must be given priority here, as compared to empirical data. Perhaps there is just a slightly longer response delay than has been predicted.

And to also be fair, I haven't corrected, extrapolated, assumed correlations at a macro level or "adjusted" anything yet. And while we are now talking about sampling across the state, obviously maybe now the entire state of Idaho can't be included in the concept of "global".

Just for fun, I might pull one temperature site from each state, and see what a low data density exercise looks like in terms of a US answer. Statistically insignificant, but it would be fun to see anyway. I mean, Hansen "corrected" the US into a warming trend....wonder what the data says, prior to aggregating, correcting, correlating without providing the details, extrapolating and aggregating? Oh, and not asking the ASA to put their stamp of approval on the statistical fundamentals.

Really? Do the Aussie have temperature records of the same quality and length as the US? Once I can get to the confidence interval ranges based on sampling density for the US, that will give me a basis for data density to instantly jump to elsewhere, with at least a stab at a similar confidence interval.

 

yeah, the Aussie discussion has always been an interesting one. North America data has always been pointed at as local and cherry picked and Australia is the globe. Not sure how that one works, and have thought about since I was told that last April. The fact is the southern hemisphere has always been warmer than the northern hemisphere, even though the population is biased in the northern hemisphere.

 

The Aussie data has been criticized for changing to warming due to homogenization. Some great locations that have not changed were changed to warming based on other stations.

 

Interesting. I found, during my peek around the climate websites on how they do things, that they provide this great little program to do just that. "Oh, everyone uses this" they say. Just on the first pass it looked like a "we will fill in the blanks for you" exercise, and as I am trying to avoid exactly those types of random "here let us all do our aggregation, interpolation, extrapolations and correlations the same way lest someone notice that the answers all become different if we don't" exercises.

 

Summary: No actual results that mean anything, but some fine conspiracy theories.

And that's why the world doesn't take you seriously.

 

Idaho Temperature Profile Using Mean of Mean Daily Temperature for a Century

So now we have the yearly mean, of the daily mean temperatures, for this group of long standing Idaho temperature datasets.

The previous graph gave you the type of graph that folks are always waving around...a single metric, no measure of uncertainty, but easy to show. This graph demonstrates the actual variability that they DON'T show you.





More importantly, this graph shows (FINALLY!) that after 3 levels of averaging we are getting to the point where the central limit theorem is showing up. 2 of the 5 statistical tests explained previously have determined this 3rd level of means are best fit with a normal, and the other 3 at least want to fit it with something Gaussian.

So now we can talk about a 95% confidence interval, for this data, without pre-processing, normalizing, adjusting, correcting, randomly correlating, interpolating or extrapolating, of approximately spanning 2.7C

Not fractions of 1F. Not fractions of 1C. And we aren't talking about a trend here...just the uncertainty around a single, common measure of central tendency. The mean can vary, year to year, by 2.7C. At this point, I don't even care about why, only that normal weather and El Nino's and snow storms and colder winters and warmer summers and everything in between can be expected to be within this range. So if you tell me you that you can find within this data, using a nth degree polynomial, some R^2 fit of means of 0.99 of +0.2C that is GREAT!!!! And I could care less, at this point in time, because it is within the normal variation of temperature in this data in this area.




JC if these don't work for images let me know and I'll get another different method to attach these things.