Inherent Predictability, Requirements on the Ensemble Size, and Complementarity

Abstract

Faced with the scenario when prediction skill is low, particularly in conjunction with long-range predictions, a commonly proposed solution is that an increase in ensemble size will rectify the issue of low skill. Although it is well known that an increase in ensemble size does lead to an increase in prediction skill, the general scope of this supposition, however, is that low prediction skill is not a consequence of constraints imposed by inherent predictability limits, but an artifact of small ensemble sizes, and further, increases in ensemble sizes (that are often limited by computational resources) are the major bottlenecks for improving long-range predictions. In proposing that larger ensemble sizes will remedy the issue of low skill, a fact that is not well appreciated is that for scenarios with high inherent predictability, a small ensemble size is sufficient to realize high predictability, while for scenarios with low inherent predictability, much larger ensemble sizes are needed to realize low predictability. In other words, requirements on ensemble size (to realize the inherent predictability) and inherent predictability are complementary variables. A perceived need for larger ensembles, therefore, may also imply the presence of low predictability.

Ensemble size: How suboptimal is less than infinity?

“…Using an idealized statistical model of the predictions, they found that statistically significant positive skill can be detected with 40 members for signal‐to‐noise ratios greater than 0.3, while with five members positive skill can only be detected for signal‐to‐noise ratios greater than 0.6. These results are consistent with those of Kumar and Hoerling () and Kumar and Chen (). Müller et al.…”

“…For seasonal forecasts, there are several studies that look at the relationship between prediction skill and ensemble size. Kumar and Hoerling () and Kumar and Chen () investigate the relationshiop in an idealized context, where predictions are represented by draws from distributions with assumed mean and variance. The deviation of the predicted distribution from the climatological distribution is interpreted as a signal‐to‐noise ratio and used to stratify results.…”

The relationship among probabilistic, deterministic and potential skills in predicting the ENSO for the past 161 years

“…Usually, it involves two branches in the field of predictability researches on the El Niño-Southern Oscillation (ENSO) prediction. The first one is to investigate the ENSO potential predictability, which will reveal what the upper limit of the prediction skill might be and how much room would leave for the improvement of ENSO prediction systems (Tang et al 2005(Tang et al , 2008Cheng et al 2010a; Kumar and Hu 2014;Kumar and Chen 2015). The common measures of the potential predictability can be categorized into variancebased metric and information-based metric, both without using the observations.…”

“…Several efforts have been devoted for investigating their relationship in the framework of the deterministic measures. For instance, it was found that the potential predictability metrics are good indicators in quantifying the deterministic actual skill in many ENSO models Kumar et al 2001;Tang et al 2008;Cheng et al 2011;Kumar et al 2001;Kumar and Chen 2015). Recently, the linkage between probabilistic actual skill and deterministic actual skill has been addressed.…”

Seasonal prediction skill and predictability of the Northern Hemisphere storm track variability in Project Minerva

“…As a result, the ensemble mean still contains a considerable amount of residual noise. On the other hand, if the intrinsic predictability is high, moderate ensemble size may be sufficient (e.g., Kumar and Chen 2015). In this case, the leading conventional and MSN EOFs are similar to each other and the former is preferable because of its simplicity (e.g., Zhu et al 2012;Shin et al 2018).…”