Sunday, July 16, 2017

Raws Warm Bias Continued

A couple of weeks ago I presented a few results from my latest project - an attempt to adjust RAWS temperature data to remove the warm bias that occurs during strong sunshine.  The goal here is to make the RAWS temperature data more useful for climate monitoring; we want to know the spatial and temporal distribution of temperature variations across Alaska, but the RAWS measurements are heavily affected by this warm bias that varies depending on sunshine and - to a lesser extent - wind speed.

In the previous post I showed the results of a bias correction based on the hourly quantity of solar radiation, for 3 different RAWS sites that are located close to reliable FAA instruments (ASOS/AWOS).  Now let's look at the effect of wind speed, which is also measured by the RAWS platform.  The charts below show the residual differences between the RAWS and FAA temperatures after the solar adjustment has been applied, with hourly mean wind speed on the horizontal axis.  The red markers indicate the median difference for each wind speed value; note that wind speed is reported to the nearest whole number in mph.

At all 3 sites, increasing wind speeds cause the RAWS temperature to decrease relative to the ASOS temperature, which is what we expect; when a breeze is blowing, the thermometer is naturally aspirated and the airflow through the thermometer housing helps reduce the artificial warming from solar heating.  This means that the warm bias becomes less of a problem as the wind picks up, and therefore it also means that my solar adjustment is too great when the breeze is blowing: if I apply my solar adjustment without regard to wind speed, then my adjusted temperatures will be too low (as shown in the scatter plots).

The obvious next step is to model the wind speed effect in a similar manner to the solar effect, and I've done that using another analytical function to describe the relationship.  After optimizing the fit of the function for each site separately, the results look like this:

Happily we find that the average wind speed dependence has been largely removed, although of course this is not a perfect process; the temperatures still seem to be biased a bit high at low wind speeds at Eagle and Lake Minchumina.

So after all this we have a set of hourly adjusted temperatures for these RAWS sites, and we can now run a test to see whether the revised data show monthly or annual climate variations that are similar to those measured at the FAA sites.  Here we are interested not so much in the long-term average bias, which can always be removed by subtracting the long-term normals, but in the sign and magnitude of month-to-month and year-to-year changes.

Ideally we would find that such changes are very similar for each pair of sites; for example, when the adjusted Fairbanks RAWS data say that a month was 3°F warmer than normal, then we want to see that the ASOS data show the same anomaly.  If this is true, then the monthly temperature differences would remain constant over time - indeed the differences would be zero if the bias is fully removed - and then we could claim that the adjusted RAWS data provide a true estimate of the long-term temperature variations.

I'll start by showing results from Lake Minchumina, where the adjustment procedure seems to have paid off handsomely.  The first chart below shows the May, June, and July monthly means of daily high temperature before and after the RAWS adjustment; the FAA/AWOS temperature is also plotted in blue.  Note that these results are drawn only from the sample I used for the adjustment process - i.e. only "peak sunshine hours", so the high temperatures might be different from 24-hour values in some cases.  Clearly the adjusted RAWS numbers show very similar month-to-month and year-to-year changes to the AWOS data.  The unadjusted RAWS data also capture the major ups and downs, but notice that there's a trend in the differences: the unadjusted RAWS line is closer to the others in more recent years.

The chart below highlights the trend issue by showing the monthly mean differences of daily high temperature between the two sites, with the unadjusted differences indicated with solid lines and the adjusted differences shown with dashed lines.

The key thing to note here is that the adjusted differences don't change significantly over time - there is little trend and the monthly variance is much smaller than for the unadjusted data.  This means that, as we saw above, the adjusted RAWS temperatures essentially move in lockstep with the AWOS temperatures.  This is in contrast to the unadjusted RAWS data, which show a remarkable trend: the RAWS warm bias has diminished considerably in the past few years.  We might be tempted to speculate about instrumentation changes as a cause for this, but the fact that the bias correction eliminates the trend suggests that solar radiation has been reduced significantly in recent years.  Obviously I'll have to confirm whether that is the case; it would be an interesting result by itself.

In conclusion, the removal of the RAWS warm bias at Lake Minchumina appears to work very well as a means to improve the quality of the data for climate monitoring.  Unfortunately, the monthly mean temperature results are not as encouraging for Fairbanks and Eagle - see below.  The long-term average bias has been removed, but the monthly temperature differences are not significantly less variable than for the unadjusted RAWS data.

The disappointing results at Fairbanks could be related to the fact that the RAWS and airport ASOS sites are over 12km apart, in contrast to Lake Minchumina where the two sites are only a few hundred meters apart.  I looked into using Fairbanks' Fort Wainwright ASOS instead of the airport, but the Fort Wainwright historical data are not as complete.

In Eagle the problem could simply be that the solar warm bias is smaller, as shown in the first post, so there's less opportunity to improve the RAWS data.

Finally, as a measure of the degree of improvement, here are (1) the correlations of the monthly mean temperature anomalies before and after adjustment, and (2) standard deviation of the monthly mean temperature differences before and after adjustment.  The higher the correlation and the smaller the standard deviation, the better.

SiteCorrelation: before (after)Standard deviation (°F): before (after)
Lake Minchumina0.93 (0.99)1.27 (0.43)
Fairbanks0.91 (0.92)1.56 (1.37)
Eagle0.98 (0.98)0.79 (0.70)


  1. From a brief search there appears to be at least three humidity and temperature sensors deployed. This list may be incomplete or inaccurate:

    The latest to be supported 39094:

    If that's the case then might there be an adjustment offset of historic data specific to the type of sensor deployed? May be worth determining what was there and when.


    1. Thanks Gary - good links. Yes, presumably different sensors are selected by different operating agencies, and changes are made over time. It would certainly make sense to tie the adjustments to specific sensors, but obtaining the historical equipment information may be very challenging. It may be the only way to really succeed with this idea, however.

    2. Of interest are the various insolation reflecting devices. There ability to normalize the sensor vs true local temps is...versus a fan-fed device...of importance.

      The area below the sensor if dark and heated could also add to the bias. Same for time since deployment dependent surface darkening via local debris like tree sap and any number of particulates that collect.

      I guess if it were my job I'd place the sensor inside a large diameter sufficiently long tube that by it's color (white/metallic/?) absorbed the least sun heat and allowed for the most vertical flow of ambient air. The angle of solar reflectance required may prevent that design however.

      The removal of bias may be sensor and location dependent.