How Good is the Reanalysis?

As regular readers here know, the NCEP/NCAR reanalysis is a very useful tool for examining historical climate conditions across the globe.  The reanalysis contains a complete estimate of the state of the atmosphere every 6 hours all the way back to 1948; here's a link to the original publication that introduced the groundbreaking project.  On this blog we've often used the reanalysis data to explore Alaska climate, for example here and here.

A question that naturally arises is, how well does the reanalysis data correspond to reality?  In the paper linked above, the authors note that variables like upper-air temperature and wind are strongly constrained by the observations, and so are generally very reliable, but other variables are more influenced by the model's internal physics, because there are few or no direct observations of these variables.  Surface temperature is an example that is somewhat constrained by the observations, but precipitation is completely determined by the model (i.e. no precipitation observations are assimilated by the model).

In view of this suggestion that the temperature and precipitation analyses might not be that great, I thought it would be interesting to look at Fairbanks data to assess the fidelity of the reanalysis.  I took the monthly mean temperature and precipitation from the nearest reanalysis grid point to Fairbanks and compared it to the observations from Fairbanks airport; the chart below show the correlations by month, with rank correlation used for precipitation.

We see that the reanalysis temperature is very good (correlation above +0.9) from October through April, but the performance drops off very dramatically in the summer.  Remarkably, the reanalysis temperature in July is almost uncorrelated with what actually happens; but interestingly August is much better.

As expected, the precipitation analysis is less good, with correlation coefficients of only about 0.6-0.7 in winter, and very low correlations in June and July.

The chart below shows the results for 3-month periods, which we might expect to fare a little better because the random daily fluctuations tend to even out.  Unfortunately this isn't true, as the correlation values are similar to those for monthly data.

The reanalysis performance is so poor in summer that I had to look at the data more closely; see below for year-by-year comparisons of the temperature and precipitation in May-June-July, which is the worst-performing season in the chart above.  The temperature chart is fascinating, because it shows that while the reanalysis generally captures the sign of many of the changes from year to year, it has a serious problem with decadal trends.  Remarkably, the reanalysis temperatures became substantially colder from the 1970s to the early 1990s, while the observations showed the reverse, and since 1995 the reanalysis has become much warmer while the observed trend has been small.  I can't say for sure what might cause this, but it probably arises from changing systematic bias in the model either when new observations are introduced (e.g. satellite data from 1969 on) or when the ocean temperature patterns shift (e.g. the Atlantic or Arctic temperatures).  After all, the reanalysis is only a model, with lots of assumptions and physical parameterizations.

The May-July precipitation comparison shows more what we would expect for a system that has very little skill; the model is simply unable to reproduce the precipitation amounts in the Fairbanks area in summer.  This isn't surprising at all, because summer precipitation in interior Alaska mainly occurs from localized convective activity or from rather small upper-level features, which are either not resolved on the coarse grid or not well-represented in the model physics.  An interesting point is that the reanalysis has shown generally wetter summer conditions since 1995, and especially since 2004, but there is less change in the observations.

It would be interesting to look at other locations in Alaska to see what the regional performance is like; and I also want to look at the more modern Climate Forecast System Reanalysis (CFSR) from 1979-present, to see if it does a better job in the warm season.  In the meantime, however, it seems we should be very cautious about using or accepting results based on the reanalysis warm-season temperature and precipitation over the Alaskan interior.