Discussion of Verification Concepts in<i>Forecast Verification: A Practitioner's Guide in Atmospheric Science</i>

Glahn, Bob

doi:10.1175/1520-0434(2004)019<0769:dovcif>2.0.co;2

Cited by 13 publications

(9 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Because of uncertainties in model formulation and in initial conditions, climate predictions in model space deviate away from the true evolution in observation space. Model predictions x should then be considered as proxy information that can be used to infer the probability of future observables y (Glahn 2004;Stephenson et al 2005;Jolliffe and Stephenson 2005). To make inferences about future observables, one needs a probability model that can give the probability p( y|x) of future observable quantities y when provided with model prediction data x, such as the model of Eq.…”

Section: Bayesian Forecast Assimilationmentioning

confidence: 99%

Toward an Integrated Seasonal Forecasting System for South America

et al. 2006

View full text Add to dashboard Cite

This study proposes an objective integrated seasonal forecasting system for producing well-calibrated probabilistic rainfall forecasts for South America. The proposed system has two components: (i) an empirical model that uses Pacific and Atlantic sea surface temperature anomalies as predictors for rainfall and (ii) a multimodel system composed of three European coupled ocean-atmosphere models. Three-month lead austral summer rainfall predictions produced by the components of the system are integrated (i.e., combined and calibrated) using a Bayesian forecast assimilation procedure. The skill of empirical, coupled multimodel, and integrated forecasts obtained with forecast assimilation is assessed and compared. The simple coupled multimodel ensemble has a comparable level of skill to that obtained using a simplified empirical approach. As for most regions of the globe, seasonal forecast skill for South America is low. However, when empirical and coupled multimodel predictions are combined and calibrated using forecast assimilation, more skillful integrated forecasts are obtained than with either empirical or coupled multimodel predictions alone. Both the reliability and resolution of the forecasts have been improved by forecast assimilation in several regions of South America. The Tropics and the area of southern Brazil, Uruguay, Paraguay, and northern Argentina have been found to be the two most predictable regions of South America during the austral summer. Skillful rainfall forecasts are generally only possible during El Niño or La Niña years rather than in neutral years.

show abstract

Section: Bayesian Forecast Assimilationmentioning

confidence: 99%

Toward an Integrated Seasonal Forecasting System for South America

et al. 2006

View full text Add to dashboard Cite

show abstract

“…This case is identical to the way that a trapezoidal ROC area is calculated given forecasts expressed as probabilities for categories [section 3a (3)]. Since the probabilities themselves are ignored when performing a ROC analysis (Mason and Graham 2002;Glahn 2004;Wilks 2006), probabilistic forecasts are reduced to forecasts of ordinal polychotomous categories. In the current context, therefore, the 2AFC score is equivalent to the standard way in which the ROC technique is performed in forecast verification, except that the probabilities associated with each point on the curve are undefined (Mason and Graham 2002).…”

Section: ) Polychotomous Forecastsmentioning

confidence: 99%

“…(7) is that the score considers only the ordering of the probabilities, ignoring the actual probability values themselves, and thus is insensitive to any monotonic transformation of the probabilities. This insensitivity is the same problem as the insensitivity of the ROC to calibration (Glahn 2004). Although 2AFC scores can be defined that explicitly consider the actual probabilities (appendix B), these scores are not proper (appendix C).…”

Section: ) Discrete Probabilistic Forecastsmentioning

confidence: 99%

A Generic Forecast Verification Framework for Administrative Purposes

Mason

Weigel

2009

Monthly Weather Review

View full text Add to dashboard Cite

There are numerous reasons for calculating forecast verification scores, and considerable attention has been given to designing and analyzing the properties of scores that can be used for scientific purposes. Much less attention has been given to scores that may be useful for administrative reasons, such as communicating changes in forecast quality to bureaucrats and providing indications of forecast quality to the general public. The two-alternative forced choice (2AFC) test is proposed as a scoring procedure that is sufficiently generic to be usable on forecasts ranging from simple yes-no forecasts of dichotomous outcomes to forecasts of continuous variables, and can be used with deterministic or probabilistic forecasts without seriously reducing the more complex information when available. Although, as with any single verification score, the proposed test has limitations, it does have broad intuitive appeal in that the expected score of an unskilled set of forecasts (random guessing or perpetually identical forecasts) is 50%, and is interpretable as an indication of how often the forecasts are correct, even when the forecasts are expressed probabilistically and/or the observations are not discrete.

show abstract

“…Model predictions are best considered as proxy information that can be used to infer the probability of future observables (Wilks 2000;Glahn 2004;Stephenson et al 2005). Because of uncertainties in model formulation and in initial conditions, climate predictions of x drift away from the observed values y.…”

Section: Bayesian Forecast Assimilationmentioning

confidence: 99%