Ensemble Methods for Neural Network‐Based Weather Forecasts

Abstract

Ensemble weather forecasts enable a measure of uncertainty to be attached to each forecast, by computing the ensemble's spread. However, generating an ensemble with a good spread‐error relationship is far from trivial, and a wide range of approaches to achieve this have been explored—chiefly in the context of numerical weather prediction models. Here, we aim to transform a deterministic neural network weather forecasting system into an ensemble forecasting system. We test four methods to generate the ensemble: random initial perturbations, retraining of the neural network, use of random dropout in the network, and the creation of initial perturbations with singular vector decomposition. The latter method is widely used in numerical weather prediction models, but is yet to be tested on neural networks. The ensemble mean forecasts obtained from these four approaches all beat the unperturbed neural network forecasts, with the retraining method yielding the highest improvement. However, the skill of the neural network forecasts is systematically lower than that of state‐of‐the‐art numerical weather prediction models.

Combining distribution‐based neural networks to predict weather forecast probabilities

“…The error growth for our results is proportionally larger than that in Scher and Messori (2021) and suggests that for Z500 the distributions produced by our approach are close to the real distributions for the 3-day hindcast, but suffer greatly as the lead time increases. Scher and Messori (2021) did not use their approach to predict T850 and thus we instead compare our T850 values with the "dressed" ERA CRPS values for the T850 5-day hindcast used by ECMWF to benchmark their operational IFS (Haiden et al, 2017(Haiden et al, , 2018. Note this is not an exact comparison because the ERA CRPS values are only for extratropical regions whereas our CRPS values are for the entire global region.…”

“…Even though we use data at a much coarser resolution, our stacked neural network approach performs much better than the approach in Scher and Messori (2021) for the 3-day hindcast but much worse for the 5-day hindcast. The error growth for our results is proportionally larger than that in Scher and Messori (2021) and suggests that for Z500 the distributions produced by our approach are close to the real distributions for the 3-day hindcast, but suffer greatly as the lead time increases. Scher and Messori (2021) did not use their approach to predict T850 and thus we instead compare our T850 values with the "dressed" ERA CRPS values for the T850 5-day hindcast used by ECMWF to benchmark their operational IFS (Haiden et al, 2017(Haiden et al, , 2018.…”

“…A limitation of the deep learning approaches used in the works referred to above is that it is difficult to infer the uncertainty of the predictions from their results (Schultz et al ., 2021). Some previous works address these limitations by using an ensemble of deep learning models to produce a probabilistic forecast (e.g., Bihlo, 2021; Scher and Messori, 2021; Weyn et al ., 2021). However, choosing a good ensemble of models is non‐trivial (Scher and Messori 2021) and may be computationally expensive because it requires the network to be trained multiple times.…”

Deep Learning for Geophysics: Current and Future Trends

“…Researchers used DL to estimate ground-level PM2.5 or PM10 levels by using satellite observations and station measurements (Li et al, 2017;Shen et al, 2018;Tang et al, 2018). DL also helps improve the accuracy of weather forecasting, which is a long-standing challenge in atmospheric science (Bonavita & Laloyaux, 2020;Scher & Messori, 2021). The tracks of typhoons were predicted with a GAN based on satellite images (Rüttgers et al, 2019).…”

“…With multiple realizations of dropout, the results are collected, and the variance is computed as the uncertainty. DL with uncertainty estimation in inference is reported in areas such as volcano-seismic monitoring (Bueno et al, 2019), geomagnetic storm forecasting (Tasistro-Hart et al, 2020), weather forecasting (Scher & Messori, 2021;Bonavita & Laloyaux, 2020), soil moisture predictions (Fang, Kifer, et al, 2020) and earthquake locations estimation (Mousavi & Beroza, 2020b).…”