Studying Temperature Data Using the Language of Science

"The earth's climate system is warmed by 35 C due to the emission of downward infrared radiation by greenhouse gases in the atmosphere (surface radiative forcing) or by the absorption of upward infrared radiation (radiative trapping). Increases in this emission/absorption are the driving force behind global warming. Climate models predict that the release of greenhouse gases into the atmosphere has altered the radiative energy balance at the earth's surface by several percent by increasing the greenhouse radiation from the atmosphere. With measurements at high spectral resolution, this increase can be quantitatively attributed to each of several anthropogenic gases. Radiance spectra of the greenhouse radiation from the atmosphere have been measured at ground level from several Canadian sites using FTIR spectroscopy at high resolution. The forcing radiative fluxes from CFC11, CFC12, CCl4, HNO3, O3, N2O, CH4, CO and CO2 have been quantitatively determined over a range of seasons. The contributions from stratospheric ozone and tropospheric ozone are separated by our measurement techniques. A comparison between our measurements of surface forcing emission and measurements of radiative trapping absorption from the IMG satellite instrument shows reasonable agreement. The experimental fluxes are simulated well by the FASCOD3 radiation code. This code has been used to calculate the model predicted increase in surface radiative forcing since 1850 to be 2.55 W/m2. In comparison, an ensemble summary of our measurements indicates that an energy flux imbalance of 3.5 W/m2 has been created by anthropogenic emissions of greenhouse gases since 1850. This experimental data should effectively end the argument by skeptics that no experimental evidence exists for the connection between greenhouse gas increases in the atmosphere and global warming."

Nope, not an experiment dude!!! Experiments don't predict, they show results. This is the same information that is used all the time. Models are not experiments.

 

Nice job of selective reading. Let me highlight the relevant part for you.

You do know what "quantitative" means, don't you?

 

doesn't mean what 120 PPM of CO2 does to temperatures.

 

Some of us certainly do. And we are discussing why, in climate science, they don't provide the uncertainty in their conclusions, instead focusing on metrics that avoid discussing the how, if's, butts.

And you are acting as an enabler for just such antics, pretending you know anything about quantitative just because you can spell the word.

 

Peakprofit I hope you understand that this has turned into a religion now, right? So going against a religion is always futile.

For me personally, I hope we are actually seeing a trend of warming, for historically it has always been a very positive thing for humanity, while the cooling has always been horrible for humanity. The hysterics here is astounding, given what any warming has always yielded for humanity. The understanding about climate change is so limited, which is proven out by the models, in which they inserted garbage in, and got flawed models, that couldn't predict (*)(*)(*)(*).


Yet the hysterics will go on, even as gov't chooses to wreck economies that feed people, even as, if the earth is indeed warming, the co2 levels will give us greater plant health, greater yields, producing more food than ever, even if we cannot afford to buy it due to the economic effects of addressing co2 levels using tunnel vision. Several scientists have suggested addressing co2 with land management, even building greenhouses next to coal fired plants to use some of that co2, for green houses are pumping in co2 in order to grow higher yield healthier plants. This means gov't and the UN are not serious about this co2 deal, and it is merely a way for a few elites to make more money, even as the common people will suffer from economic problems.


If you think you will change the minds of this new religious cult, you are pissing into the wind. There will be books written on the psychology of the hysterical, long after they have returned to the dust of the earth, as well as sociological studies.


The red flag was when it was uttered from lying lips........the science is settled. Can you imagine how real scientists must have shuddered when that was uttered? Even if their own paychecks were coming from grants, in the billions, that were given in order to support the IPCC conclusions,, prior to the reseach being done? LOL This is Twightlight Zone material here. It would have made a good show in that old series.

 

A religion founded on science, how cool. If only all of our beliefs could be subjected to the scientific method.

 

As someone who spotted peak oil as a religion (or cult, or belief system, or whatever) early on, I would say that I recognize religious aspects among the zealots of the position, I do not necessarily agree that those scientists who understand that science is never settled, are still out there, plugging away at better understandings, narrowing the uncertainties involved, making sure they do their due diligence.

Those who decided many years ago to corrupt science to their own ends will fail, and ultimately the religious aspects will slowly disappear among the acolytes as science wins.

That is the history of it, yes. Plus don't forget Canada, if it wasn't for global warming, we wouldn't have Pamela Lee or really, really good ice hockey players in the Americas.

All models are wrong, some are useful. The trick is determining which is which.

As a scientist, it was irrelevant if I changed minds, and still is. I am not advertising, I am not a pimp, I do not make a choice between honest and trying to convince people to follow me like lemmings to the sea. This is what the climate folks were talking about years ago, and I object to that corruption of the basic precepts of doing science, or being a scientist. This idea operates in both directions, I do not care if I convert climate folks away from their zealotry, nor do I care if my work demonstrates that the world will end tomorrow in fire and flame, the answer is the answer, the uncertainty in the answer is provided with the answer, and those who corrupt the process through their advocacy or ego or desire to be a bigger academic name in whatever specialty they are in are not doing justice to the spirit or intent of the profession.

Not a red flag. An admission that whoever uttered those words knows NOTHING about science.

I certainly did.

 

Progressive don't truly believe in global warming. They just see it as a tax revenue.

 

it's funny how mother nature didn't give in to it!! Oh Crap they say!!!!

 

They don't say "OH CRAP!". That implies the cognitive ability to determine that their dogma might be suspect. When an acolyte is operating inside a belief system, only the information allowed to be discussed can be discussed. Any information brought into the system in unedited/uninterpreted/undigested fashion doesn't generate the reaction of "Oh Crap!", oh no. The proper response is to scream "DENIER!!" and accept praise for being the first to notice.

So, knowing that this is the case, you line up a different challenge, such as this one. You allow them to pick one of their pre-approved references, preening and chuckling all the way to their latest cut and paste forum win, and do the same thing that the hammer wielding executioner does to the happy cow coming down the chute to slaughter, never knowing what hit them.

We are talking about SCIENCE, not this ridiculous forum game of "I'm right your wrong..denier denier denier" crap. If it isn't done to that standard, I can take it apart on that basis. If it is done to that standard, in the text or references will be either the CYA language or assumptions that are presented as known, but aren't. The acolytes aren't taught to THINK about the preapproved information, they certainly aren't schooled in the ways of science and allowed to think about it for themselves, imagine how ugly that might get. Folks might pull an "average joe" stunt ..look at temperature records, how well the models work, and start laughing out loud, right there in the pews.

Can't have laughter in the pews, it is unseemly among the true believers. They might get ideas....can't have that either...

 

Let us now talk about uncertainty with real data.

So we examined how mean monthly temperatures within an entire country can fluctuate, and the odds of how much they fluctuate, based on nothing more than variation within a month. And how that level of uncertainty alone is far higher than what was provided as a “confidence interval” within the data, and worse YET, how little of the data could be expected to be within the interval quantified…FROM THE PEOPLE WHO PROVIDED THE DATA no less.

But an average for a country is itself a massive aggregation of a bunch of different stations, averaged yet again by month, averaged yet again to get a country wide answer. And while the theory of statistics is great and wonderful and statisticians love just crunching data, if fundamental assumptions don’t hold true, then many of the conclusions drawn from erroneous assumptions (such as that of normality) just cannot be made, from a statistical point of view.

So now we are going to look at one station. I went here:

http://www.ncdc.noaa.gov/cdo-web/datatools/findstation

And collected all data from about 1914 or so, right on through a few days before Christmas, 2014. The data comes from this station:

ABERDEEN EXPERIMENT STATION ID US GHCND:USC00100010

First we had to do a little cleanup however, because as with all things “data” and sometimes discussed (or not) by the author, is how to handle the busts. The dataset contained 36,436 temperature records, starting on June 1, 1914. Ending on Dec 21, 2014. -9999 appears to be used as a null to designate no reading, 100 records were eliminated for having a TMax of -9999, there were then an additional 71 with a TMin of -9999 that were eliminated as well. I then eliminated one other data point, that on March 1, 1928. When complete, the data we are going to examine has 36,264 records.
Here is a graph of the mean temperature at this one station spanning the past century.



Quite amazing how CO2 has dramatically changed the temperature profile of this particular station in Idaho. Of course, we are now dealing with data at the daily level, and not what happens when it has been "fixed", "corrected", "adjusted". This means that we have removed some assumptions, but not all (such as individual temperature station quality). And certainly I am not going to provide a picture of melting glaciers to provide the right "visual" to make sure the audience begins to lean in my direction before I even mention the results. We are going to assume, at this station, people have been taking temperatures correctly and consistently, completely ignoring yet ANOTHER level of uncertainty related to sample quality itself. Notice that this ITSELF is an assumption, exactly the kind of CYA statement I’ve pointed out within the original climate papers. I, as an ex-scientist, reserve the right to do the same thing, and tell you, the reader, about it. If you disagree, we can discuss how you might go about correcting the temperature difference between...say...how Bob records temperature readings, versus Joe. Certainly this type of error is in here, but that is not today's issue. We can tackle that issue later, and why we need to do it is ultimately expressed best with a picture.



So how about we just examine the data.So first let us do this…we will take all months from 1914 to present, put them in chronological order and look at the distribution of maximum temperatures across a century. No parlor tricks of confidence intervals, I describe each and every month right here, by individual probability density functions just as I described in a prior post. For the record, I used my favorite metric to parameterize the distribution, the Anderson-Darling test. Should someone want to use their own statistical test on the data, have at it, I told you where to get the data and how I’ve assembled it, if you think your test is superior to nailing down the particulars of the distribution, knock yourself out. As a real scientist, at some point in the future I might do it myself just to see if it changes underlying conclusions.

Here are all the January’s from 1914 to 2014.

Here are all the February's from 1914 to 2014

Here are all the March's from 1914 to 2014

Here are all the April's from 1914 to 2014



Charts continued in next post.

 

Part II - Continuation of Charts


Here are all the May's from 1914 to 2014

Here are all the June's from 1914 to 2014

Here are all the July's from 1914 to 2014

Here are all the August's from 1914 to 2014

Here are all the September's from 1914 to 2014

Here are all the October's from 1914 to 2014

Here are all the November's from 1914 to 2014

Here are all the December's from 1914 to 2014


I think I will make no further comment to allow all these horrifying visuals of how global warming has changed the very fabric of temperature at this Idaho station to sink in, before we analyze it any further. Comments on trends folks THINK they see are always appreciated, even though, as some who have fled the scene might point out, eyeballing it isn’t sufficient in the world of science (not that THEY would know) but eyeballing it IS the first thing a scientist does when the data rolls out of the computer and onto the screen or paper. Consider this that data before we really find out what is being hidden in the woodpile.

 

So where's the TOBS correction?

Oh my. You left that out? No wonder you get garbage results.

And you didn't shift to anomolies before accounting for dropouts. Even more errors in your data now.

Did you look at station history? Probably not. That means more errors.

Was there any UHI correction? Did you check?

This is a problem you see so often with denier amateurs. Being they're so unfamiliar with the basics and so unwilling to learn from good science, they end up remaking a lot of the mistakes that scientists found and fixed many years ago. Then, based on their own junk results, they declare anyone who got good results is a fraud.

So, why do you fuzzify the data with the "variation within a month" nonsense? Looks like a parlor trick, or maybe you just don't understand how averages and errors are defined by standard statistical methods.

 

When you begin an investigation using raw data, the first thing you DON'T do is begin correcting, adjusting or multiplying that data by whatever other numbers happen to strike your fancy.

I don't have garbage results. I have empirically described temperature readings, month by month, for about a century. They are designed to put bounds and ranges on the variation of the data, that we might understand the range within these things happen.

Are you saying that you have studied the given daily data and reached a different conclusion? Feel free to describe your process, and if you have an interest in the exact distributions I use to describe these ranges, feel free to ask.

You are mistaken. I haven't baselined any of this to create an analysis by anomaly. I'm not even sure I like the convention of creating a baseline, and then measuring everything as an anomaly against it. I've tried it but haven't posted on it yet. And if you are saying that the data used...daily temperatures at a particular station has errors in it, may I ask if you have personal experience with HOW these errors came about? Are you saying that people have changed the data for some reason? Decided to be inconsistent for some reason, because they were having a bad day? A tummy ache perhaps? To date, I haven't complained about the DATA at all, I have only concentrated on describing it, and trying to understand why this obvious and visible ranges are not accounted for when others begin to aggregate this data without properly quantifying the natural variability. If you are claiming that the variability is all man made, well that would be quite a statement, which is why I ask if you have personal experience with it.

Of course, if temperature data is just manufactured and people are making up numbers, then that would invalidate just about all earthly temperature records and I would need to begin studying how uncertainty is quantified by satellites.

Define "look at". I certainly examined a century worth of temperature data provided. And if your point is that the range in the data is even WIDER than I have quantified, well that just makes all of this analysis an underestimate...and we have to worry about the "noise" in the system being higher than +/- 10C, which is the size of the 50% confidence interval I have currently prepared for my next post.

I used the data provided by NOAA for that particular station. Are you claiming that they did not provide data, but have already manipulated the provided information?

Found and fixed? Can you refer me to empirical distributions of uncertainty in the seminal temperature aggregation work? Can you show why the experts assume normality when it isn't evident within the data? Can you provide the aggregation matrix used to put any two stations together within an aggregation? I mean, if this has all been published then I can use it and compare THAT to data as well.

If you do not understand how to bootstrap into an empirical distribution for a given block of data (such as the variation within a month) I recommend you reread my explanation for how I have already done it, and you can use this as a general reference as well.

http://en.wikipedia.org/wiki/Bootstrapping_(statistics)

Maybe you don't understand the difference between the basic statistics I've already explained, and employed, and parlor tricks. If you are saying I have done anything incorrect in terms of analyzing the data provided, please, I am all ears. If you want to use other tests to parameterize the results, fine, feel free to demonstrate the difference between your distributions and mine. If you think NOAA is now passing out manipulated data, and you have an inside source for the 36,000 CORRECT temperature data points, by all means let me have them, or analyze them yourself and tell us how much of a difference it made when compared to my distributions.

I recommend first you wiki up "parlor" tricks, and then "bootstrapping" in statistics, and learn the difference between the two. For starters.

Don't be discouraged, it only took me the better part of a decade as a working scientist to be able to do all these things easily, I'm sure you are much smarter than I, and with my help, if might only take you half a decade.

 

The first thing you do is correct known and quantifiable errors in the data.

You refuse to correct known and quantifiable errors in the raw data. Hence, garbage out.

Here's a source to help you learn about the TOBS correction.

http://variable-variability.blogspot.com.au/2012/08/a-short-introduction-to-time-of.html

You liking it is not relevant to the necessity of it.

Temperatures vary a lot spatially. Anomalies do not vary much spatially. If you have a certain anomaly in New York, you know it will be nearly the same from Ohio to Maine. Observation of the physical world has shown us that.

If data is missing, it can be filled in quickly and accurately by using anomalies from neighboring sites. Not doing so will lead to much bigger errors, being temperatures vary so much over time. Failure to infill leads to big errors. A couple missing days in August will pull the average way down if the data isn't infilled. The use of anomalies allows that.

Instead of spending your time thinking up new conspiracies, it would perhaps be better spent learning the basics of the science.

You're using "variability" in a novel way. Congratulations on your new math. However, why should anyone else care about it?

You'll find satellite data has to be twiddled much more than surface data. A microwave measurement has to be processed a lot more to turn it into a temperature reading, in comparison to reading a thermocouple or column of mercury.

I actually laughed, that was so senseless.

And that's why nobody wants to talk to you.

I'm afraid I can't provide you with any info to support your strange use of statistics. My suggestion is that, if you're unwilling to write a book telling the world how it's gotten all the statistics so wrong, is that you do some of your own research on how and why adjustments are done. We can only hold your hand to a certain extent. You can get started here.

http://data.giss.nasa.gov/gistemp/

 

That depends on the resolution of the data you are dealing with, and if those errors are known. For example, within a +/- 10C error bar for a 50% confidence level, and you have TOBS effect being measured in the +/- 0.3C range, it seems reasonable to establish that independently just as I am the natural variation in the provided data.

As for the errors that cannot be quantified, so you recommend we do as the papers that started this debate did....and simply say that since we don't know them, we won't count them? When the errors are known to be greater than 0, you cannot make a better answer by pretending they do not exist, and yet this is EXACTLY the path taken in both papers mentioned.

You do not get to pretend that all errors must be removed when the peer reviewed science papers are perfectly happy to do exactly what I have. And have already explained in prior posts, related to fundamental assumptions in data.

You refuse to understand that when quantifying uncertainty, you generally narrow from the largest downward...and you cannot even claim that a TOBS correction of +/- 0.3C can be SEEN within the the ranges already quantified. This is called science, and is as fundamental as significant digits when estimating the precision on basic engineering calculations. Are you now claiming that climate scientists don't understand these basic concepts?

Please prove your statement using data. I have already demonstrated how temperature varies not spatially but just between days in the month. It strikes me as intuitive that they would also vary spatially, and when I get to comparisons between datasets and calculations of correlation we can discuss that part, in the appropriate language. You use the word "anomalies" in a way that does not appear to match what has been claimed by those who wrote up the temperature data supplied for Germany.

I would be more than happy to use their methods to calculate "anomalies" (already have really) and it does not appear to support your statement. So please feel free to prove it...and calculation of anomaly within the Idaho data would be fine, how you determined it within the given natural uncertainty, and then how you correlate it with any other station within, say, 500 miles? Just one correlation and how you calculated it as a demonstration should suffice to both establish your understanding of the language and prove your statement.

Given time, I will certainly get there as well, and then we can compare answers.

Certainly someone may have CLAIMED that, and you might be parroting them, but until I get to the second temperature station, within a recognized grid size in Idaho, and someone provides or I calculate the correlation matrix between multiple stations, you can CLAIM whatever you like. Feel free to prove it if you'd like as well, and then I can compare my answer to yours.

Please reference the correlation matrix used to make this happen. Are these anomalies perfectly positively correlated? Almost perfectly positively correlated? Sorta correlated? For your statement to be true implies this matrix exists...and yet you TALK but don't show it. I would love to see it. As just one example, can you tell me about the correlation between Aberdeen Idaho and any other stations in Idaho? Who checked the internal consistency of the matrix? How was it checked? Was it done by a meteorologist somewhere, or did statisticians finally get involved and offer up the missing 3rd party independent review of it? I can answer every one of those questions on the last multi-correlated stochastic model I built, and name the independent review organization and professionals involved. Why do you come to this conversation claiming these things exist, and yet haven't demonstrated a single correlation? They must be out there, for your statement to be true, and while I assume you are generally ignorant of science I do NOT assume you would just lie for the fun of it.

I find that comment wildly amusing. Let me know when you understand how to bootstrap into a distribution...ANY distribution...and then we can discuss what it will take to get you away from that pacifier that prevents you from speaking the language.

 

It doesn't have a +/- 10 C error bar. Your use of the variation of the distribution instead of the variation of the mean ... as Pauli supposedly put it "Das ist nicht nur nicht richtig, es ist nicht einmal falsch!".

Yes, the imaginary errors you say are there, but can't describe. You're right, I don't know how to account for them, being that they're imaginary.

And you're badly mistaken. Whether a temperature distribution is gaussian or not simply doesn't matter. Sure, it matters to your mistaken use of variance of the distribution, but it matters not to variation of the mean.

I'm pointing out you're failing hard at basic statistics. Do you really not understand the difference between variation of the distribution and variation of the mean of the distribution?

No. I'm done with doing the basic research that you refuse to do, apparently as a point of pride. You're making extraordinary claims, so you better have extraordinary evidence. You don't have any evidence other than your very peculiar statistics, and that's why nobody is paying any attention to you.

 

Worauf immer du Bock / Lust hast

 

Show me your confidence interval around this stations temperature data (pick your month and year) and I'll show you mine. The trick to your statement is that you didn't provide your confidence interval, whereas I have certainly quantified mine, and understand that you can't even make the statement you just did without it.

We are discussing data at this point, and a mean is a function of that data. However, the mean is also provided on every graphic provided, and you can see the empirical variation around the mean itself, assuming you have eyes. Are you saying that this empirical distribution of means doesn't match yours? Reference yours please...it isn't hard...here is the wiki on how to calculate it.

http://en.wikipedia.org/wiki/Mean

Now you have come off the rails, or have been struck stupid. Because it is one of those questions at the HEART of the matter. Please provide your analysis showing that the skewness in your distributions is 0, of either empirical monthly data, or the means themselves, that I may present my opposing data showing that not ONCE in thousands of distributions have I yet found this mythical beast.

Funny that the "peer reviewed science" assumes normal Gaussian distributions though, isn't it?

I have not provided the variance of any distribution of temperature within the Idaho data, only the range of certain percentiles containing 90% of the data. Those whiskers describe certain percentiles, and you cannot calculate the variance from the information provided on those box whisker plots. If you wish to discuss the variance at the monthly level sure...I can do that as well. Of what use do you consider that to be?

Now run off and educate yourself on variance.

http://en.wikipedia.org/wiki/Variance

And within a single distribution, there is NO variation around the mean, it is derived from a given set of data, and without varying the data any further (such as th difference between sampling with or without replacement) it does not change. So your statement is the equivalent of 2+2=4...yes....thank you for stating a definition you can memorize, and most of us learned in grade school. Did you also know that the sun usually rises in the east?

Come back and say that after you learn what variance is. And how the shape of any distribution is critical to understanding what assumptions can be made when aggregating data, or CANNOT be made.

 

That would be a neat trick. How do you calculate the confidence interval from a single data point?

 

Now, you need to show that Aberdeen Idaho is fully representative of the planet earth.

This would be HUGE, of course, as it would let us stop sending up orbiting monitors in space, stop measuring ice, ocean temperatures, and other ground based monitors the world over.

I'll be interested in seeing you justification for making this gigantic cost savings.

 

HADCET is considered to be a good proxy for North American and that is the longest temperature record existing located in England. If neither place shows the warming that is and has been predicted by the alarmists, then where will they go to get warmer temperatures? You realize, don't you, that the 'warmest' year is warmer by 1/100ths of a degree than 2010 based on calculating that on surface temperatures that change and have temperatures constantly adjusted by 'scientists' and that the only reason it is warmer is because of a spot in the North Pacific?

 

A month of data is not a single data point. Because of the way I have bootstrapped each month separately, I can choose a confidence level for each month based on the percentiles of the distribution, and convert the percentile to confidence, by month, expressed in degrees C. It would also be completely reasonable to NOT do it that way and open up the interval even more, based on the data density issue alone. But I am trying to keep things simple here on the forum, so for now I'll just stick with empirical everything.

 

Can't do that until I can quantify the whole planet, and I am doing this thread as an interesting exercise in testing the assumptions, aggregations, summaries, confidence intervals and everything else I can do relatively easily with the computing power and time available to me.

If I had to pick a side today, I might go the other way. If the uncertainties at ground level are unmanageable, the signal to noise ratio too low (such speculation not to be confused with proof of any kind), it might be better to just go with Argos probes and satellites and get rid of all the human induced issues altogether. Maybe some heavily QC'd network of minimal size to itself QC the satellites?

 

The mean temperature of Aberdeen Idaho in any given month or year is a single data point.

What you have done is calculated the amount of noise in a given system while eliminating any long term trends. It's no wonder you can't find any warming trend.