Wednesday, May 24, 2017

Medium-Range Model Performance

There's been some chatter among meteorologists in recent days about unusually poor forecast performance by the U.S. Global Forecast System, a model that predicts the global circulation for weather forecasts out to 15 days.  Summer is a more challenging time for weather forecasts in general, because relatively unpredictable small-scale features are more important than in winter.  But even compared to normal for the time of year, the recent performance has been poor.

The pair of maps below shows a 7-day forecast of 500mb height anomaly (departure from normal) on the left and the ensuing verification on the right; the forecast was issued on May 14.  While some of the features were correctly anticipated, the forecast was badly wrong from easternmost Siberia to Alaska and also in western Europe.  The spatial correlation of anomalies ("anomaly correlation") was only 0.26.


Here's the same pair of maps but for the 7-day forecast from ECMWF.  It's widely known than ECMWF's data assimilation and modeling methods produce generally superior weather forecasts compared to GFS, but in this case the difference was dramatic.


To put this event in context, I calculated the daily anomaly correlations over the Northern Hemisphere north of 20°N for 7-day forecasts back to early 2016, and I also looked at Environment Canada's global model forecasts (labeled as "CMC" in the charts below).  The first chart shows the dramatic drop-off in skill in recent days for both GFS and CMC; both models have suffered the same fate, but ECMWF has remained relatively unscathed.  Note, however, that ECMWF is not without fault - it had a remarkable failure last summer.


If we plot the differences between the models, we see that the ECMWF forecasts have outperformed more dramatically in recent days than at any other time in the past year or so.


Zooming in on the North Pacific region (see below), there is of course more noise in the statistics, but similar patterns are evident.  The CMC model has not fared quite as badly as GFS in recent days, but again the ECMWF has performed much better than the other two models.



In conclusion - let's hope the current episode proves to be instructive for the model developers so that the science can continue to advance.  NOAA scientists are paying attention to these forecast skill "dropouts" - see for example the article beginning on page 5 of the following newsletter:

https://www.jcsda.noaa.gov/documents/newsletters/2017_02JCSDAQuarterly.pdf

The bigger picture from the results I've shown here is that modern global circulation models now have the ability to predict weather patterns out to 7 days or more with fairly good skill on average, and I find that to be a remarkable achievement.

Saturday, May 20, 2017

Does Wet=Cool in Summer? The Fairbanks Evidence

Hi, Rick T. here. On Friday's ACCAP monthly Alaska Climate Outlook Webinar, the question came up afterwards about whether the Climate Prediction Center's June through August 2017 outlook of both a warm and wet tilt in the odds over a large part of mainland Alaska was reasonable, on the idea that if it's a rainy (dry) summer, then would not it be more likely to be cool (warm)? So I thought it would be worth reprising that question here on Deep Cold.

Below is the basic Fairbanks evidence in the form of a scatterplot. For summer precipitation I'm using the Standardized Precipitation Index (SPI). The SPI is widely used in drought assessment  and is extremely useful for comparing precipitation anomalies for regions with quite different climates (say, Fairbanks vs. Yakutat). However, it's also a handy way of characterizing anomalies at a single locations, since the SPI  effectively takes into account the distribution of precipitation amounts, which very often do not follow a bell (Gaussian) curve and turns the anomalies into standard deviations. For temperatures, which at the monthly and seasonal scale usually do follow a bell-curve distribution (in the absence of trend), standardized anomalies are just the observed minus the long term average, and that difference divided by the long term standard deviation.When we work-up those statistics for each summer during the Weather Bureau/NWS era at Fairbanks (using the full 1930-2016 record as the baseline), here's what we get. The summer SPI is on the vertical axis and the summer temperature anomaly is on horizontal axis. The red dashed line is the linear correlation, which is significant, and does lend support to the idea that warm summers (right-hand side of the plot) do trend to be dry, while cool summers due tend to be wet.

That's all well and good, but with 87 data points it's kind of hard to make out the details. Since we're interested in the warm/dry and cool/wet idea, here's the same plot, but I've masked out all the points in the "near normal" category (using the three-category characterization, so ±0.43 standard deviations) and colored the quadrants that correspond to the four combinations of significantly wet/dry/warm/cool.
From this it is much clearer that there have been summers that have been both warm and wet overall (e.g. just last year, upper right quad) and summers that have been cool and dry (e.g. 1947, lower left quad), but clearly wet and cool or warm and dry are much more favored. It's very hard to read in the plot, but by this measure there have been no "cool and dry" summers (lower left quad) in Fairbanks since 1969. Upon reflection, this isn't  surprising, since we all know there is a very marked trend in June through August average temperatures.
We can make some account of the current climate by using a more recent baseline period for temperatures (there is no multi-decadal trend in summer rainfall). Since the Climate Prediction Center is basing the outlook on the 1981-2010 reference period, we'll use that. We can also adjust the correlation between rainfall and temperature to take into account the temperature trend (by using generalized least squares regression). When we make those changes, we get this:
So we see the overall correlation between summer temperature and precipitation is weaker (but still significant) and that the modern criteria of what a warm and wet summer wound be really are an unusual combination. Come back in September to see how the outlook worked out.

Update: A couple of folks have asked about how wildfire activity fits into this, so here's the basic temp vs. rainfall anomaly plot with the million acre plus seasons highlighted. In this view, only two seasons, 2002 and 2015, had significantly above average summer rainfall in Fairbanks and also burned a million acres or more (statewide).


Thursday, May 18, 2017

Not So Cold in Barrow

The sun is now remaining above the horizon for 24 hours a day on the Arctic coast of Alaska, but temperatures are only just getting up to about the freezing point.  Barrow (or Utqiaġvik) has not yet exceeded 32°F this spring, although it was 36°F back on January 1st!  The normal date to break the freezing mark for the first time in Barrow is May 9th; temperatures have been less variable than usual of late.


Looking back at the 2016-2017 winter, it's remarkable to note that Barrow failed to reach -40° for the 5th consecutive winter; the lowest temperature observed all winter was -36°F, which is exactly the same as the last two winters.  The two winters before that only saw -39° and -38°.  The persistent absence of deep cold is quite remarkable.  In terms of daily maximum temperatures, not a single calendar day remained at or below -20°F in the past winter, and it's the first time that has happened in the modern history of daily observations from Barrow.


On another note, here's an interesting chart showing the date of the first spring day with a temperature of 40°F or higher in Barrow.  The absence of variability in the 1950s and 1960s is rather striking, and the two outlying early occurences in 1931 and 1936 really stand out.  There seems to be no particular reason to doubt that it really did get that warm in late April in those years; it was very warm in Kotzebue too.  The scanned observer forms are below (click to enlarge; note "rain" on April 23, 1931).



Monday, May 15, 2017

More on Spring Warming Trends

Some readers may find this post a bit esoteric, but I think the end result is worth reporting.  My interest was piqued by Rick Thoman's chart that I posted last week, showing the dates on which 65°F or warmer was first observed each year in Fairbanks, dating all the way back to 1905.  The early data may be a bit dubious, but it's interesting to look at the complete series.  As I noted before, the chart shows that the earliest "first warm" days have become much earlier over time, but there is essentially no trend in the latest 10% of the distribution - there have been several instances of very delayed warmth in recent decades.  This implies that not only has the climate warmed at this time of year, but inter-annual variance has increased considerably.


The question that comes to mind here is, "Is the increase in year-to-year variance statistically significant?"  Visually it's clear from the chart that the early decades saw more consistency in the arrival date of summer-like warmth in Fairbanks, whereas recent decades have been more variable.  But given a warming climate, could random chance have produced a chart like this?  Or can we conclude that there really is something going on here in terms of the year-to-year variability?

To address this question, I created 1000 simulations of the daily temperature history in Fairbanks for the full 113-year period.  First I calculated the daily normals for overlapping 30-year periods throughout the history, i.e. the daily mean and standard deviation of daily maximum temperatures, and then I produced random Gaussian simulated data constrained by these normals and the observed lag-1 autocorrelation coefficient (which also varies through the year).  Finally, for each simulated history I pulled out the annual dates of first 65+°F and then performed the same trendline regressions as shown in Rick's chart.

Here's an example of what I'm talking about.  The first chart below shows the observed dates from the actual data - the same dates as Rick plotted, although there's a minor difference because of the way leap years are handled.  The upper and lower trendlines diverge, as Rick showed.  The next two charts show two of the simulations; in the first, the trendlines show the kind of slope we might expect in a warming climate, but in the second, the trendlines happen to converge.  Both simulations use the same climate normals as the real history, and the differences are just caused by random chance.




Now we come to the interesting part.  Based on the full sample of 1000 simulations, the charts below show the distribution of trendlines for the 10th, 50th, and 90th percentiles.  On each chart I've marked the category in which the actual observed slope falls; so for example, the observed 10th percentile (early arrival of summer) has advanced much more than most of the simulations.  This means that the slope of the lower trendline in Rick's chart is much steeper than we would expect based on the underlying climate normals.  In contrast, the middle (50th percentile) trendline is not far off where we would expect it to be; and the 90th percentile (late arrival of summer) has advanced less than we would expect.




The final step in my analysis was to look at the distribution of the difference between the 90th and 10th percentile trendlines; in the actual history of Fairbanks the lower and upper tails have diverged, but what about the simulations?  The chart below shows that nearly all of the simulations have a smaller difference in slopes; less than 1% of the simulations have a divergence as large as in the real world.  In fact, it's more common for the trendlines to converge than to diverge, as the upper trendline tends to be steeper than the lower trendline (this can be seen in the charts above).


So what do we conclude?  The bottom line is that the observed increase in year-to-year variability in timing of the "first warm day" appears to be highly statistically significant.  Simulations based on a Gaussian model of daily temperature variations show a less than 1% chance that the observed divergence in the upper and lower trendlines could have occurred by random chance.  We can also say that recent decades have seen both more unusually early arrivals of warmth and more unusually late arrivals of warmth than would be expected; if the variations were just random, we probably would have seen less change on the warm side and more change on the cool side.

Of course, all of this depends on the assumption that my Gaussian simulation model is realistic; we know the temperature distribution in Fairbanks is skewed in spring, but exploring whether that makes a significant difference here is a topic for another day.

Tuesday, May 9, 2017

River Ice Update

Break-up is essentially complete now on the Tanana and Kuskokwim Rivers and is under way on the Yukon River; the Yukon ice moved out on May 3 at Dawson, May 4 at Eagle, and today at Circle.  These dates are very close to long-term normals.  The tripod at Nenana fell on May 1, which is also about normal, but Bethel's break-up guessing game ended on May 6, a few days ahead of normal.


As we've noted before, the long-term trend towards earlier break-up is less statistically significant at Bethel than at Nenana.  There are two reasons for this: first, the (assumed linear) rate of change itself is smaller at Bethel, and also the inter-annual variance is greater at Bethel.  However, the early break-ups of recent years (2014-2017) have been much more unusual at Bethel compared to long-term norms.  The reason for this is probably the proximity of Bethel to the persistent and very unusual warmth in the northern Pacific Ocean.


While we're on the topic of variance and spring warming, here's a very interesting chart that Rick Thoman sent over, showing the long-term history of the first date of 65°F or higher in Fairbanks.  The three trendlines represent linear regression applied to the 90th percentile (blue), the median (green), and the 10th percentile (orange) of dates.  While the trend lines for median and early (10th percentile) warmth have advanced over time as expected, the 90th percentile has barely changed.  This implies that the variance has increased quite substantially.  While this could perhaps be a statistical quirk of this particular temperature threshold, it's an interesting result that deserves further investigation.



Wednesday, May 3, 2017

Howard Pass Study

This week I'm in Anchorage at the Climate Prediction Application Science Workshop, and among many other interesting presentations I noted a poster by Ken Hill and Pam Sousanes on weather at Howard Pass.  On this blog we've looked at extreme winds and wind chill at Howard Pass on several occasions.  A reduced-size photo of the poster is below, and here's a link to a full-size pdf (2.5MB):

Sunday, April 30, 2017

Spring Warming Trends (continued)

In my last post I suggested that annual dates of river break-up in Alaska may be able to provide quantitative information about long-term temperature changes at this time of year; this is possible because of the high degree of correlation between break-up dates and spring temperatures.  The annual date of "green-up" in Fairbanks can function in the same way as a climate marker.

After pondering the topic for a couple more days, it now seems clear that the method I used in the first post (read it here) was far from robust.  There's surely no significant difference between a correlation of (say) -0.86 and -0.87, so it's probably not reasonable to pick out the precisely optimal correlation and then claim to have identified a "true" temperature trend.  Accordingly, I think it's probably just coincidence that the results for Fairbanks and Nenana lined up so closely.

Not being one to give up, however, I moved on to a different approach by considering what range of temperature trends could be consistent with the observed correlation between temperature and break-up (or green-up) dates.  This is something that can be addressed with statistical simulation; so I produced 1000 simulated histories of southeast interior April-May temperatures by generating values based on the observed correlation with break-up date.  Each of the simulated histories has approximately the same correlation with break-up date as the actual reported temperature history, but the trends differ widely owing to the random component of the simulations.  Note that I'm assuming there are no other long-term changes that have systematically affected the break-up dates one way or the other.

Here are a couple of examples: the top chart shows the break-up dates and a simulated history that happens to have a very low (near zero) temperature trend, and the second chart shows an example with a very high temperature trend.



For both of these examples, the correlation of the simulated temperatures with break-up dates is very close to the correlation observed in reality, so these are outcomes that "could have" happened based on the physical connection between break-up and temperature.  However, both of these are unlikely outliers; the chart below shows the full distribution of trends from the 1000 histories, plotted as a cumulative distribution function.


It's nice to see that the 50th percentile of simulated trends lies almost exactly on the reported trend, so we can say that the changes in break-up at Nenana are entirely consistent with the temperature trend reported in the southeast interior climate division.  Bear in mind that the simulation process has no knowledge of the actual temperature trend; we have backed it out from the break-up dates.

Here's the same chart derived from green-up dates and April-May temperatures in Fairbanks.

In this case the simulated trends do not line up perfectly with the reported trend, as about 60% of the simulations have more warming than reported by the thermometers.  This is consistent with the simplistic result in my first post, but now we can quantify the probability that the trend is higher than reported.  Based on these results, and if my assumptions are correct, it is about 60% likely that Fairbanks has warmed more than the official climate record indicates (based on a 1974-2016 linear trend line).

Finally, here's an interesting result based on break-up dates of the Koyukuk River in Bettles.


Here we find that nearly all of the simulated histories have less warming than the reported April-May temperatures from the Bettles observing site; it appears to be 90% likely that Bettles has over-reported the warming trend.  In fact, more than 50% of the simulated trends are negative, and when we look at the break-up dates (see chart below) it becomes immediately obvious why this is the case: the linear trend-line for break-up dates shows a very slight increase over this period (although several years are missing near the beginning).  The lack of change in break-up date appears to be inconsistent with the reported warming at Bettles; more investigation is required.