Deep learning for post-processing ensemble weather forecasts

Abstract

Quantifying uncertainty in weather forecasts is critical, especially for predicting extreme weather events. This is typically accomplished with ensemble prediction systems, which consist of many perturbed numerical weather simulations, or trajectories, run in parallel. These systems are associated with a high computational cost and often involve statistical post-processing steps to inexpensively improve their raw prediction qualities. We propose a mixed model that uses only a subset of the original weather trajectories combined with a post-processing step using deep neural networks. These enable the model to account for non-linear relationships that are not captured by current numerical models or post-processing methods. Applied to the global data, our mixed models achieve a relative improvement in ensemble forecast skill (CRPS) of over 14%. Furthermore, we demonstrate that the improvement is larger for extreme weather events on select case studies. We also show that our post-processing can use fewer trajectories to achieve comparable results to the full ensemble. By using fewer trajectories, the computational costs of an ensemble prediction system can be reduced, allowing it to run at higher resolution and produce more accurate forecasts. This article is part of the theme issue ‘Machine learning for weather and climate modelling’.

Towards implementing artificial intelligence post-processing in weather and climate: proposed actions from the Oxford 2019 workshop

“…Gronquist et al . [71] demonstrate applying DL methods to substantially improve uncertainty quantification skills for global weather forecasts, including for extreme weather events.…”

“…It is in use by several national centres in meteorology, including the US National Center for Atmospheric Research (NCAR) and the UK Met Office (UKMO), as well as The Alan Turing Institute (the UK's national institute for data science and artificial intelligence) and the British Antarctic Survey (BAS), among others. Note that parallel data archive efforts are underway in other communities, including Environnet [99], Weatherbench [100] (), Spacenet () and various authors who make their datasets public [71] among others.…”

Opportunities and challenges for machine learning in weather and climate modelling: hard, medium and soft AI

“…Another promising approach is the use of machine learning in uncertainty quantification. This includes both the use of approaches to represent uncertainty within models, for example via the use of multi-bin Markov chains, GANs [31], or the use of machine learning techniques to post-process the output of ensemble simulations [32,33].…”

Addressing up-scaling methodologies for convection-permitting EPSs using statistical and machine learning tools

“…Machine learning, as well as deep learning algorithms and neural networks are becoming increasingly popular and widespread throughout the scientific world for their natural quality of self-adjustment and self-learning (Bremnes, 2019;Grönquist et al, 2021). It is likely that such algorithms will find more and more applications in NWP (Dueben and Bauer, 2018).…”

“…It is likely that such algorithms will find more and more applications in NWP (Dueben and Bauer, 2018). Tree-based ensemble methods have recently been successfully applied to improve the skill of low visibility conditions in an operational meso-scale model (Bari and Ouagabi, 2020), while Krasnopolsky and Lin (2012) demonstrated how machine learning algorithms can decrease the bias for low and high rainfall.…”