Best Practices for Postprocessing Ensemble Climate Forecasts. Part I: Selecting Appropriate Recalibration Methods

Abstract

This study describes a systematic approach to selecting optimal statistical recalibration methods and hindcast designs for producing reliable probability forecasts on seasonal-to-decadal time scales. A new recalibration method is introduced that includes adjustments for both unconditional and conditional biases in the mean and variance of the forecast distribution and linear time-dependent bias in the mean. The complexity of the recalibration can be systematically varied by restricting the parameters. Simple recalibration methods may outperform more complex ones given limited training data. A new cross-validation methodology is proposed that allows the comparison of multiple recalibration methods and varying training periods using limited data.Part I considers the effect on forecast skill of varying the recalibration complexity and training period length. The interaction between these factors is analyzed for gridbox forecasts of annual mean near-surface temperature from the CanCM4 model. Recalibration methods that include conditional adjustment of the ensemble mean outperform simple bias correction by issuing climatological forecasts where the model has limited skill. Trend-adjusted forecasts outperform forecasts without trend adjustment at almost 75% of grid boxes. The optimal training period is around 30 yr for trend-adjusted forecasts and around 15 yr otherwise. The optimal training period is strongly related to the length of the optimal climatology. Longer training periods may increase overall performance but at the expense of very poor forecasts where skill is limited.

Robust skill of decadal climate predictions

“…Nevertheless, the ensemble mean signal may be highly skilful if it correlates with the observed variability, and its magnitude can be adjusted in a post-processing step to produce realistic forecasts. 19,29,31,32 If the signal-to-noise paradox applies to decadal predictions then previous studies [12][13][14][15][16][17] may have underestimated the skill. This is because the ensemble size may have been too small to remove the noise, or the skill measure used may have penalised errors in the magnitude of the predicted signal which could potentially be corrected before issuing forecasts.…”

“…As noted above, the magnitude of forecast anomalies must be adjusted if the signal-to-noise ratio is incorrect. Although several objective methods have been proposed 19,29,31,32 further work is needed to establish the best approach. We, therefore, follow the previous studies 18,20,21 and simply compare forecasts and observations using standardised anomalies (Fig.…”

Application of Postprocessing to Watershed-Scale Subseasonal Climate Forecasts over the Contiguous United States

“…A second strategy to improve climate forecast skill lies in the statistical postprocessing of dynamical forecast model outputs. Postprocessing is applied through statistically translating raw, large-scale dynamical model outputs to a regional scale that is useful for local applications, in this case regional water managers (Sansom et al 2016;Li et al 2017). Raw dynamical model output typically requires postprocessing or downscaling (a form of postprocessing) to be used in follow-on applications due to systematic biases, unreliable ensemble spread, and/or forecasts' lack of skill.…”

Parametric decadal climate forecast recalibration (DeFoReSt 1.0)

“…NGR extends traditional model output statistics (MOS, Glahn and Lowry, 1972) by allowing the predictive uncertainty to depend on the ensemble spread. A further extension, proposed by Sansom et al (2016), also accounts for a linear time dependency of the mean bias. However, CCR is closely related to NGR in that the forecast mean error and forecast spread are jointly corrected to satisfy the necessary criterion for reliability that the time mean ensemble spread equals the forecast root mean square error.…”