Sequential Aggregation of Probabilistic Forecasts—Application to Wind Speed Ensemble Forecasts

Abstract

In numerical weather prediction (NWP), the uncertainty about the future state of the atmosphere is described by a set of forecasts (called an ensemble). All ensembles have deficiencies that can be corrected via statistical post‐processing methods. Several ensembles, based on different NWP models, exist and may be corrected using different statistical methods. These raw or post‐processed ensembles can thus be combined. The theory of prediction with expert advice allows us to build combination algorithms with theoretical guarantees on the forecast performance. We adapt this theory to the case of probabilistic forecasts issued as stepwise cumulative distribution functions, computed from raw and post‐processed ensembles. The theory is applied to combine wind speed ensemble forecasts. The second goal of this study is to explore the use of two forecast performance criteria: the continuous ranked probability score (CRPS) and the Jolliffe–Primo test. The usual way to build skilful probabilistic forecasts is to minimize the CRPS. Minimizing the CRPS may not produce reliable forecasts according to the Jolliffe–Primo test. The Jolliffe–Primo test generally selects reliable forecasts, but could lead to issuing suboptimal forecasts in terms of CRPS. We propose to use both criteria to achieve reliable and skilful probabilistic forecasts.

A pre-registered short-term forecasting study of COVID-19 in Germany and Poland during the second wave

“…This is done separately for incident and cumulative forecasts. The inverse-WIS ensemble is a pragmatic strategy to base weights on past performance, which is feasible with a limited amount of historical forecast/observation pairs (see 42 for a similar approach).…”

Short-term forecasting of COVID-19 in Germany and Poland during the second wave – a preregistered study

“…This is done separately for incident and cumulative forecasts. The inverse-WIS ensemble is a pragmatic strategy to base weights on past performance which is feasible with a limited amount of historical forecast/observation pairs (see Zamo et al 2020 for a similar approach).…”

Prediction with expert advice applied to the problem of prediction with expert advice

“…PWEA is a well-studied and successful framework and set of methods, and, given its applicability to the novice/2-expert problem, it is plausible that it will be helpful for policy makers who also need to form judgments based on expert advice. Various variants of it have already been applied successfully to numerous real-world contexts including forecasting wind speed (Zamo et al, 2020), subseasonal meteorology forecasting (Brayshaw et al, 2020), and disease progression (Morino et al, 2015). Let us work through the details of how a real policy maker might use it.…”