Common Issues in Verification of Climate Forecasts and Projections

Abstract

With increased interest in climate forecasts and projections, it is important to understand more about their sources and levels of skill. A starting point here is to describe the nature of the skill associated with forecasts and projections. Climate forecasts and projections typically both include time varying forcing of the climate, but only forecasts have initial conditions set close to the observed climate state. Climate forecasts therefore derive skill from both initial conditions and from forcing. The character of the initial condition skill and forcing skill is different. Skill from initial conditions results in a narrowing of expectations relative to a climatological distribution and points toward a more favoured part of the distribution. Forcing skill could result from a shift in the preferred parts of the climatological distribution in response to forcing, or it could result from a shift in the entire distribution, or both. Assessments of forcing skill require time averages of the target variable that are long enough so that the contributions from internal variations are small compared to the forced response. The assessment of skill of climate forecasts and projections is inherently partial because of the small number of repeated trials possible on typical climate time scales but is nonetheless the only direct measure of their performance.

Evaluation of ECMWF-SEAS5 Seasonal Temperature and Precipitation Predictions over South America

“…Despite numerous advances and constant development of climate modeling, errors are intrinsic to the process. However, several studies point to different causes for the deviations, such as deficiency in SST simulation, errors in the initialization of soil moisture conditions, and inappropriate physical parameterization [3,[34][35][36]78]. In addition, improved extreme events prediction requires a deep understanding of drought and flood mechanisms, refined observations from data assimilation, better parameterizing techniques, efficient ensemble methodologies, and proper uncertainty quantification [17].…”

“…Seasonal climate forecasts start from an observed state of all Earth system components and then evolve over a few months. Thus, errors present at the beginning of the forecast persist or grow during the integration of the model, reaching magnitudes comparable to the forecast signals [34][35][36]. In this scenario, the coupled general circulation model components must be consistent with each other at the initial time of the forecasts to avoid the influence of initialization shock, which is associated with the departure of the model climatology from the observed [37].…”

“…Furthermore, other errors in climate predictions are systematic, and by comparing retrospective forecasts (hindcasts) and observations, it is possible to correct these systematic flaws [34][35][36]. These systematic errors, also known as biases, arise from the difference between the atmospheric and oceanic states simulated by the model and those of the observed climate and can vary by season, region, and forecast lead time.…”

“…In this context, we note that evaluating the performance of forecasts and hindcasts is essential since the latter has a more extensive set of predictions, allowing the model's assessment with more robust statistics. On the other hand, the retrospective predictions present more idealized conditions than the prognostic forecasts since the hindcasts are better calibrated and submitted to complete data ingestion for the initial conditions [34][35][36].…”

Concurrently assessing water supply and demand is critical for evaluating vulnerabilities to climate change

“…Starting in 2006, the Washington State Legislature mandated that a “forecast” be released every 5 years by the OCR to provide the most current analysis of the forces influencing future water availability in eastern Washington to inform OCR's development of water supplies for instream and out‐of‐stream uses (RCW, 2006). Though our analysis technically provides projections, not forecasts (sensu Risbey et al., 2022), we call it the Forecast (capitalized) as this is the term used in the legislative mandate and, therefore, our agency partner's preferred and published term (Hall et al., 2022).…”