Friday, July 3, 2015

How Good is the Reanalysis?

As regular readers here know, the NCEP/NCAR reanalysis is a very useful tool for examining historical climate conditions across the globe.  The reanalysis contains a complete estimate of the state of the atmosphere every 6 hours all the way back to 1948; here's a link to the original publication that introduced the groundbreaking project.  On this blog we've often used the reanalysis data to explore Alaska climate, for example here and here.

A question that naturally arises is, how well does the reanalysis data correspond to reality?  In the paper linked above, the authors note that variables like upper-air temperature and wind are strongly constrained by the observations, and so are generally very reliable, but other variables are more influenced by the model's internal physics, because there are few or no direct observations of these variables.  Surface temperature is an example that is somewhat constrained by the observations, but precipitation is completely determined by the model (i.e. no precipitation observations are assimilated by the model).

In view of this suggestion that the temperature and precipitation analyses might not be that great, I thought it would be interesting to look at Fairbanks data to assess the fidelity of the reanalysis.  I took the monthly mean temperature and precipitation from the nearest reanalysis grid point to Fairbanks and compared it to the observations from Fairbanks airport; the chart below show the correlations by month, with rank correlation used for precipitation.

We see that the reanalysis temperature is very good (correlation above +0.9) from October through April, but the performance drops off very dramatically in the summer.  Remarkably, the reanalysis temperature in July is almost uncorrelated with what actually happens; but interestingly August is much better.

As expected, the precipitation analysis is less good, with correlation coefficients of only about 0.6-0.7 in winter, and very low correlations in June and July.

The chart below shows the results for 3-month periods, which we might expect to fare a little better because the random daily fluctuations tend to even out.  Unfortunately this isn't true, as the correlation values are similar to those for monthly data.

The reanalysis performance is so poor in summer that I had to look at the data more closely; see below for year-by-year comparisons of the temperature and precipitation in May-June-July, which is the worst-performing season in the chart above.  The temperature chart is fascinating, because it shows that while the reanalysis generally captures the sign of many of the changes from year to year, it has a serious problem with decadal trends.  Remarkably, the reanalysis temperatures became substantially colder from the 1970s to the early 1990s, while the observations showed the reverse, and since 1995 the reanalysis has become much warmer while the observed trend has been small.  I can't say for sure what might cause this, but it probably arises from changing systematic bias in the model either when new observations are introduced (e.g. satellite data from 1969 on) or when the ocean temperature patterns shift (e.g. the Atlantic or Arctic temperatures).  After all, the reanalysis is only a model, with lots of assumptions and physical parameterizations.

The May-July precipitation comparison shows more what we would expect for a system that has very little skill; the model is simply unable to reproduce the precipitation amounts in the Fairbanks area in summer.  This isn't surprising at all, because summer precipitation in interior Alaska mainly occurs from localized convective activity or from rather small upper-level features, which are either not resolved on the coarse grid or not well-represented in the model physics.  An interesting point is that the reanalysis has shown generally wetter summer conditions since 1995, and especially since 2004, but there is less change in the observations.

It would be interesting to look at other locations in Alaska to see what the regional performance is like; and I also want to look at the more modern Climate Forecast System Reanalysis (CFSR) from 1979-present, to see if it does a better job in the warm season.  In the meantime, however, it seems we should be very cautious about using or accepting results based on the reanalysis warm-season temperature and precipitation over the Alaskan interior.


  1. Off topic but...10:30 pm 7/6/15 and it's still +85F at the surface. Amazing bubble of hot air above? Never in 50 years have I've experienced this length and degree of heat in Interior Alaska/Fairbanks. Needs to be documented and analyzed.


    1. Agreed, Gary. Extraordinary conditions and I'll try to put them in historical context.

    2. After some thought it does pertain to topic. Maybe after the summer a reanalysis vs reality for 2015 would be a worthwhile exercise. It may be that this is the new climate for Alaska?

      Meanwhile smoke and lowered visibility at the surface drifts around over Alaska depending upon winds. I wonder how deep/height the smoke gets? Some pilots report the smoke tops at 10K MSL plus:


  2. This is an extremely interesting analysis Richard. I am curious as to the correlations of the North American Regional Reanalysis (NARR) and the ERA-Interim Reanalysis. Both of those are more robust (incorporation of more observational data, more layers, and smaller grid cells) but only have data back to 1979. Thank you for looking into this.