Probabilistic Precipitation Forecast Skill as a Function of Ensemble Size and Spatial Scale in a Convection-Allowing Ensemble

Abstract

Probabilistic quantitative precipitation forecasts (PQPFs) from the storm-scale ensemble forecast system run by the Center for Analysis and Prediction of Storms during the spring of 2009 are evaluated using area under the relative operating characteristic curve (ROC area). ROC area, which measures discriminating ability, is examined for ensemble size n from 1 to 17 members and for spatial scales ranging from 4 to 200 km.Expectedly, incremental gains in skill decrease with increasing n. Significance tests comparing ROC areas for each n to those of the full 17-member ensemble revealed that more members are required to reach statistically indistinguishable PQPF skill relative to the full ensemble as forecast lead time increases and spatial scale decreases. These results appear to reflect the broadening of the forecast probability distribution function (PDF) of future atmospheric states associated with decreasing spatial scale and increasing forecast lead time. They also illustrate that efficient allocation of computing resources for convection-allowing ensembles requires careful consideration of spatial scale and forecast length desired.

Ensemble forecasts of a flood‐producing storm: comparison of the influence of model‐state perturbations and parameter modifications

“…It is sufficient to be able to estimate accumulations to better than 10% for almost the full length of the simulations. This result is consistent with Clark et al (2011a), which found that convective scale ensembles of nine members had similar skill to the full ensemble of 17 members for the the median ROC curve of probabilistic quantitative precipitation forecasts of 6-hourly accumulations. The standard deviation of the anomalies is remarkably consistent with the error of the mean when estimated using sampling theory (Figure 4), suggesting that the members are indeed independent and identically distributed.…”

On the forecast skill of a convection-permitting ensemble

“…In order to produce a reliable probabilistic forecast, the individual ensemble member forecasts should be equally likely to occur and cover the range of future states. Following Clark et al (2011), the ideal number of ensemble members is dependent on the point of diminishing returns, i.e., the ensemble size where no new information can be expected by additional members.…”

“…To name but a few, there are the COSMO-DE-EPS (Consortium for Small-scale Modeling -EPS, Gebhardt et al, 2011;Peralta et al, 2012;Ben Bouallègue et al, 2013;Kühnlein et al, 2014) at the Deutscher Wetterdienst (DWD), the CP version of UK Met Office's MOGREPS (Met Office Global and Regional Ensemble Prediction System, Bowler et al, 2008;Caron, 2013;Hanley et al, 2013;Tennant, 2015), a storm-scale ensemble forecast (SSEF) run by the Center of Analysis and Prediction of Storms (CAPS) at the University of Oklahoma (Xue et al, 2007Clark et al, 2011;Schumacher et al, 2013;Schumacher and Clark, 2014), WRF-based CP ensemble at NCAR (e.g., Schwartz et al, 2015), and AROME-EPS (e.g., Bouttier et al, 2012) developed at Météo France. A common feature of all of these EPSs is that their horizontal mesh size is equal to or less than 4 km, but mostly between 2 and 3 km.…”

“…They mainly focus on the investigation of the impact of CP ensemble configurations, for example, the generation of IC perturbation, representation of the model error, uncertainties from the lateral boundary conditions (LBCs), ensemble size, and spatial scale (Kong et al, 2006;Clark et al, 2009Clark et al, , 2011Bouttier et al, 2012;Ben Bouallègue et al, 2013;Kühn-lein et al, 2014;Schwartz et al, 2015;Schumacher and Clark, 2014;Romine et al, 2014;Tennant, 2015). There are few comprehensive studies on the evaluation of CP EPS, in particular, in comparison with the mesoscale regional EPS.…”

Evaluation of the Plant–Craig stochastic convection scheme (v2.0) in the ensemble forecasting system MOGREPS-R (24 km) based on the Unified Model (v7.3)

“…While progress is continually being made in improving the accuracy of single forecasts -through improvements in the model formulation as well as increases in grid resolution -a complementary approach is the use of ensembles in order to obtain an estimate of the uncertainty in the forecast (Buizza et al, 2005;Montani et al, 2011;Buizza et al, 2007;Bowler et al, 2008;Thirel et al, 2010;Yang et al, 2012;Zhu, 2005;Abhilash et al, 2013;Roy Bhowmik and Durai, 2008;Clark et al, 2011;Tennant and Beare, 2013). Of course, ensemble forecasting systems themselves remain imperfect, and one of the most important problems is insufficient spread in ensemble forecasts, where the forecast tends to cluster too strongly around rainfall values that turn out to be incorrect.…”