https://arstechnica.com/science/2023/11/ai-outperforms-conventional-weather-forecasting-for-the-first-time-google-study/
It's not always easy to distinguish between hype and reality when it comes to claims of new technological advances. Is the enthusiasm justified in this case? Are weather forecasts about to see a revolutionary step forward in accuracy? I'll offer a few words of my own perspective, as someone in the "weather business".
First, there's no doubt that AI and Machine Learning (closely related concepts) have made amazing strides for weather prediction in the past few years. In early 2021, ECMWF (the world's leading "traditional" weather forecasting organization) published a "road map" for the future of AI weather forecasts, anticipating a rather gradual pace of innovation and change; but it turned out that several leading technology companies achieved remarkable success in just the next 2 years. Crucially, the latest AI models were suddenly revealed as being able to match or even beat the ECMWF's forecast accuracy according to some metrics. ECMWF wrote about it here:
The headline result - exceeding the ECMWF's basic skill level - is a big deal. It means these new models are legitimate competitors, and it's truly remarkable that the AI scientists have achieved this so quickly, with relatively early-stage, experimental models. In contrast, the accuracy of traditional forecast models has developed steadily but very slowly over the years, relying on bigger and faster computers, more satellite data, and incrementally better physics in the models. Modern weather forecast accuracy is a great scientific accomplishment; but now without warning it is being equaled (by some metrics) with an immature technology that presumably has a lot of room to improve.
Here's an example figure from ECMWF showing gradual improvement of skill for predicting Northern Hemisphere 500mb height at various lead times. After all this work, the idea that an upstart new technology can suddenly jump in with similar or better skill is quite surprising and perhaps difficult to swallow!
But notice that I alluded to specific metrics that show the new models in a favorable light, and this is because the first generation of AI models has significant limitations. For example, it seems the models don't do particularly well with extreme events, because they are guided by (constrained by) the historical data rather than the laws of physics. It seems to me that models trained purely on historical data will always struggle in this way, but hybrid statistical-dynamical models are an obvious extension that would be more likely to handle unprecedented events.
Another area where the new models aren't yet fully capable is in terms of handling uncertainty. Forecast centers like ECMWF have become adept at running traditional models in a way that encapsulates and predicts uncertainty, and this is extremely important for valuable real-world forecasting: we need to know the plausible range of possible outcomes. In contrast, the first-generation AI models just give a single answer to the question: "given today's weather, what will the weather be N days from now?"
It's also worth noting that - at least for now - the new AI models rely on the traditional models to provide the initial conditions for the forecast, i.e. to specify what "today's weather" is in great detail around the globe. This "initialization" process is itself a tremendous scientific achievement, requiring a very advanced model to "assimilate" data from the entire observing system and create a single best guess of what's happening at every location.
What Difference Does It Make?
It's worth asking what practical difference there will be for weather forecasts created with AI models rather than traditional models. This is of course difficult to foresee, but I suggest the answer is "not much" until the mainstream weather industry fully comes to grip with the new technology and builds it into the forecasting process. The AI models have been developed by big technology companies (e.g. Google and NVIDIA) that are not (yet) in the business of selling forecasts, and the vast majority of meteorologists and meteorological scientists work elsewhere in government, academia, and traditional private sector weather companies. These two "worlds" will need to come together if the new technology is to be deployed widely, and the process won't be quick or painless.
In terms of specific predictions, I can see two things happening. First, the big-tech AI models will be licensed to private weather companies who find value in the AI forecasts and can build products for customers. The leading forecast centers like ECMWF will also build their own AI models and provide the results to users, and so the industry will gradually adapt to the new source of information; but the traditional methods certainly won't be ditched any time soon.
Second, I think the AI methods will be used to develop much better forecasts for some specific high-impact problems that are not handled well by traditional models. An example might be fire weather forecasts: for example, an AI model trained on past weather-driven fire events could provide powerful guidance for future risk. The deadly fires in Hawaii and California in recent years might have been much better predicted with a specific application of the new technology, allowing aggressive early evacuations in those rare and dangerous weather situations.
I'd be glad to hear comments and insights from readers. What else do AI methods have to offer that physics-driven "deterministic" models can't provide? What other forecast problems are poorly handled by today's usual weather guidance and might be particularly amenable to historical/statistical methods like AI?
No comments:
Post a Comment