Last year I wrote a post on the accuracy of the NCEP global
reanalysis temperatures and precipitation for Fairbanks, and it turns out that
a paper has recently been published by UAF researchers on a similar topic. Lader et al (2016) looked at 5 different
reanalysis products and attempted to assess their accuracy for statewide
historical temperature and precipitation. The
general conclusion is that there is no one reanalysis that is universally best, as each one has different biases, but for specific purposes it is possible
to identify which products are superior; for example, NASA’s MERRA reanalysis
appears to be the best for wildfire concerns (late spring and summer interior temperature and
precipitation).
An aspect of the paper that caught my eye was that they used
an independent high-resolution gridded analysis of Alaska temperature and
precipitation as the “ground truth” for measuring the error in the various
global products. (They also separately looked
at station observations, so the overall assessment was based on both sources of
verification data.) I had not previously
come across the high-resolution gridded data, so I downloaded it from NOAA (source) and took a quick look. The development of this data was described in a paper last year by Hill et al.
The maps below show the 1981-2009 average temperature for
January and July according to the high-resolution analysis. (The data extends from 1961 to 2009 at 2 km
resolution.) The January map appears to show some kind of discontinuity along the Alaska-Yukon border, which is
troubling; I can’t imagine what would cause this. Other than that, however, the analysis looks
reasonable and of course it shows lots of nice spatial detail; it is
fundamentally based on the PRISM model, which is a state-of-the-art statistical
technique for estimating localized climate variations, particularly in regions of complex terrain (see e.g. here and here).
To look more closely at the local accuracy of the gridded
analysis, I pulled out the closest grid points for Fairbanks airport and for
the Keystone Ridge co-op site. There is more than 1100' elevation difference between these two sites, so it is interesting to see if the analysis adequately
represents the temperature differences. The charts below show the January and July mean temperatures at the two sites for the common periods of record in both gridded and observed data.
For Fairbanks airport, the correspondence between modeled and observed temperatures is very good in January, with almost no bias and a high correlation, but the analysis is much less good in July, with a correlation coefficient of only 0.64 - and the gridded analysis is more than 1.5°F too cold on average. At Keystone Ridge there is a January warm bias of nearly 4°F over the short period of record, but the correlation is virtually perfect in both January and July. It's worth noting too that the mean temperature difference between the two sites is only about 0.5°F different from reality, both in January and July - so the vertical gradient of temperature is correctly represented.
The correlation between gridded and observed monthly temperatures for the two locations is shown below. We see that July is the worst performing month at Fairbanks airport (matching what we saw for the global reanalysis), but the correlation is almost perfect for Keystone Ridge throughout the year. The latter result is interesting, because the starting point for the 2-km analysis was the PRISM 1961-1990 normals, which would not have included any data from Keystone Ridge. However, Hill et al (2015) added observed monthly anomalies to the PRISM normals, so they obviously included the Keystone Ridge data; and it would seem that no other stations were close enough to dilute or modify the Keystone Ridge signal - otherwise the analysis would not show a perfect fit at that location.
Presumably the reason that Fairbanks AP fails to show a perfect fit is that there are many other nearby valley-level stations, all reporting slightly different anomalies from each other month by month, and when many of them are combined in an objective analysis scheme, the result is less than perfect at any one of the stations.
A better test of the gridded data would be to compare results for a station whose data did not make it into Hill's analysis, but we would need to see his list of stations in order to know where to start.
In conclusion, a first look at this high-resolution gridded historical data leaves me impressed and eager to see this kind of product undergo further development. It would be nice to see the Alaska PRISM normals updated for a more modern era (rather than 1961-1990) by taking advantage of the wealth of RAWS data that is now available, and it would also be great if a near-realtime update could be achieved. Anyone looking for a graduate thesis project?
Sounds like the Hill paper just compares one model (his) with other models (the various reanalysis). To say that a "ground truth" is being used is stretching it.
ReplyDeleteAs far as the discontinuity, from what I've read, the PRISM model uses a Digital Elevation Model (DEM) to interpolate parameter values. Not all DEMs are created equal. Alaska recently finished a high resolution of the state. Mostly likely the Canada DEM is of lower resolution and thus has more spatially averaged values.
There is an important difference between the models: reanalyses are created from weather forecast models (full 3-D physics) constrained by mostly satellite and 3-D observations (in many cases no surface observations at all), whereas the 2-km analysis is just a statistical analysis of surface observations. So I think it's fair to use the latter as a "best guess" of actual surface conditions.
DeleteThe words "ground truth" were mine. Lader says "As in all observational datasets, the Hill et al. (2015) data have biases, but they serve as a baseline with which one can easily compare the different reanalyses."
FYI Hill wasn't a co-author on Lader et al so he's not the one doing the comparison.
Interesting point about DEMs - thanks for pointing that out. However, the Alaska PRISM normals were created in 2000 or earlier, and I don't think Hill used topography in his interpolation, so I doubt the new Alaska DEM comes into play here.
The Alaska NPS has an agreement with the PRISM group to update the maps. The timeline and details are TBD, but it should be a great improvement.
ReplyDelete(http://depts.washington.edu/pnwcesu/search/results_detail.php?project_mod_identifier=P14AC01244/0).
Wow, Ken, that is fabulous news! The abstract looks outstanding and I eagerly look forward to the results. Thank you for letting us know about this.
Delete