Evaluation of Probabilistic Quality and Value of the ENSEMBLES Multimodel Seasonal Forecasts: Comparison with DEMETER

Abstract

The performance of the new multimodel seasonal prediction system developed in the framework of the European Commission FP7 project called ENSEMBLE-based predictions of climate changes and their impacts (ENSEMBLES) is compared with the results from the previous project [i.e., Development of a European Multimodel Ensemble System for Seasonal-to-Interannual Prediction (DEMETER)]. The comparison is carried out over the five seasonal prediction systems (SPSs) that participated in both projects. Since DEMETER, the contributing SPSs have improved in all aspects with the main advancements including the increase in resolution, the better representation of subgrid physical processes, land, sea ice, and greenhouse gas boundary forcing, and the more widespread use of assimilation for ocean initialization.The ENSEMBLES results show an overall enhancement for the prediction of anomalous surface temperature conditions. However, the improvement is quite small and with considerable space-time variations. In the tropics, ENSEMBLES systematically improves the sharpness and the discrimination attributes of the forecasts. Enhancements of the ENSEMBLES resolution attribute are also reported in the tropics for the forecasts started 1 February, 1 May, and 1 November. Our results indicate that, in ENSEMBLES, an increased portion of prediction signal from the single-models effectively contributes to amplify the multimodel forecasts skill. On the other hand, a worsening is shown for the multimodel calibration over the tropics compared to DEMETER.Significant changes are also shown in northern midlatitudes, where the ENSEMBLES multimodel discrimination, resolution, and reliability improve for February, May, and November starting dates. However, the ENSEMBLES multimodel decreases the capability to amplify the performance with respect to the contributing single models for the forecasts started in February, May, and August. This is at least partly due to the reduced overconfidence of the ENSEMBLES single models with respect to the DEMETER counterparts.Provided that they are suitably calibrated beforehand, it is shown that the ENSEMBLES multimodel forecasts represent a step forward for the potential economical value they can supply. A warning for all potential users concerns the need for calibration due to the degraded tropical reliability compared to DEMETER. In addition, the superiority of recalibrating the ENSEMBLES predictions through the discrimination information is shown.Concerning the forecasts started in August, ENSEMBLES exhibits mixed results over both tropics and northern midlatitudes. In this case, the increased potential predictability compared to DEMETER appears to be balanced by the reduction in the independence of the SPSs contributing to ENSEMBLES. Consequently, for the August start dates no clear advantage of using one multimodel system instead of the other can be evidenced.

Demonstrating the value of larger ensembles in forecasting physical systems

“…Like the Lorenz system (Lorenz, 1963), the MooreÁSpiegel system arises in the context of thermal convection but in the case of a stellar atmosphere. The Poincare' section of the MooreÁSpiegel system near the origin (an unstable fixed point) lead Balmforth and Craster (1997) to argue that the dynamics of this system can be related to those of the Lorenz system. At these parameter values, the forecast systems for MooreÁSpiegel exhibits variations in the growth of forecast uncertainty as a function of position in state space Machete, 2008), a property shared with models of the Earth's weather (Palmer and Zanna, 2013) and the Lorenz system.…”

Observationally based analysis of land–atmosphere coupling

“…However, much of the model improvements so far have been obtained over ocean, where extensive availability of observations allowed model progresses and reliable application of assimilation techniques (Rosati et al, 1997;Alessandri et al, 2010Alessandri et al, , 2011. In contrast, forecast performance over land is substantially weaker compared to the ocean (Wang et al, 2009;Alessandri et al, 2011).…”

Improvement of grand multi‐model ensemble prediction skills for the coupled models of APCC/ENSEMBLES using a climate filter

“…For example, Weisheimer et al . (2009) investigated the prediction skill for a grand MME, which is a combination of DEMETER (Palmer et al ., 2004) and ENSEM-BLES (Weisheimer et al ., 2009;Alessandri et al ., 2011). They show that the prediction skills for the grand MME are yet relatively limited.…”

“…In this work, we apply the concept of the climate filter (Lee et al ., 2011) for potential improvement of a grand MME, derived from a combination of APEC Climate Center (APCC) MME seasonal prediction system (Lee et al ., 2009) and ENSEMBLES (Weisheimer et al ., 2009;Alessandri et al ., 2011) in order to explore whether the methodology can improve the skills of the grand MME, constituent MMEs, and individual models.…”