Diagnostic Verification of the Climate Prediction Center Long-Lead Outlooks, 1995–98

Abstract

The performance of the Climate Prediction Center's long-lead forecasts for the period 1995-98 is assessed through a diagnostic verification, which involves examination of the full joint frequency distributions of the forecasts and the corresponding observations. The most striking results of the verifications are the strong cool and dry biases of the outlooks. These seem clearly related to the 1995-98 period being warmer and wetter than the 1961-90 climatological base period. This bias results in the ranked probability score indicating very low skill for both temperature and precipitation forecasts at all leads. However, the temperature forecasts at all leads, and the precipitation forecasts for leads up to a few months, exhibit very substantial resolution: low (high) forecast probabilities are consistently associated with lower (higher) than average relative frequency of event occurrence, even though these relative frequencies are substantially different (because of the unconditional biases) from the forecast probabilities. Conditional biases, related to systematic under-or overconfidence on the part of the forecasters, are also evident in some circumstances.

A skill score based on economic value for probability forecasts

“…For σ = 0.1 the forecasts resemble long-lead seasonal forecasts (Wilks, 2000), deviate only rarely from their mean value, and thus have the least potential to be informative. For σ = 0.3 the forecasts resemble shortrange weather forecasts (Murphy & Wilks, 1998), are comparatively sharp, and forecast large deviations from the mean value fairly frequently.…”

Evaluation of bias-correction methods for ensemble streamflow volume forecasts

“…Such a post-hoc correction developed using a verification data set is often referred to as calibration. Several calibration methods include the use of the conditional mean given the forecast (Atger, 2003), the use of a linear regression between probability forecasts and observations (Stewart and Reagan-Cirincione, 1991;Wilks, 2000), or more sophisticated approaches using the rank histogram (Hamill and Colucci, 1997). An advantage of post-hoc calibration is that it can minimize both unconditional and conditional biases.…”

“…A corrected ensemble volume is obtained from interpolation between the order statistic pairs. Note that similar approaches have been used by Leung et al (1999), Wood et al (2002), and others to correct biases in temperature and precipitation forecasts from atmospheric models for hydrologic forecasting.…”

“…Frequency analysis is then applied to the ensemble traces, producing probability distribution forecasts for streamflow variables. Other recent examples using ensemble prediction techniques in streamflow forecasting and water resources decision-making include Georgakakos et al (1998), Hamlet and Lettenmaier (1999), Carpenter and Georgakakos (2001), Faber and Stedinger (2001), Kim et al (2001), Yao and Georgakakos (2001), Hamlet et al (2002), Wood et al (2002), Franz et al (2003), Souza Filho and Lall (2003), Clark and Hay (2004), Grantz et al (2005), and Roulin and Vannitsem (2005), among others.…”

Watershed Response to Bias-Corrected Improved Skilled Precipitation and Temperature under Future Climate - A Case Study on Spencer Creek Watershed, Ontario, Canada

“…Two probabilistic verification measures, namely the Brier skill score (BSS) and the rank probability skill score (RPSS), mostly used in the assessment of meteorological forecasts [34][35][36], were used in this study to assess the quality of bias-corrected climate model simulated precipitation and temperature time series. The BSS and RPSS are based on the Brier score (BS) and the rank probability score (RPS), respectively.…”