On judging the credibility of climate predictions

Abstract

Incorporating a prediction into future planning and decision making is advisable only if we have judged the prediction's credibility. This is notoriously difficult and controversial in the case of predictions of future climate. By reviewing epistemic arguments about climate model performance, we discuss how to make and justify judgments about the credibility of climate predictions. We propose a new bounding argument that justifies basing such judgments on the past performance of possibly dissimilar prediction problems. This encourages a more explicit use of data in making quantitative judgments about the credibility of future climate predictions, and in training users of climate predictions to become better judges of credibility. We illustrate the approach using decadal predictions of annual mean, global mean surface air temperature.

Towards Handling Uncertainty in Prognostic Scenarios: Advanced Learning from the Past

“…The problem of uncertainty and horizons of credibility 1 of predictions of future behavior of Earth's climate system has attracted a growing interest as a consequence of the increasing demand for incorporating information about future climate into planning and decision making (e.g., IPCC 2007: FAQ 1.2, FAQ 8.1;NSF 2012;IPCC 2013: Box 11.1;Otto et al 2015). Numerous scientific institutions, including IIASA, use a variety of complex integrated assessment models to generate a great number of prognostic scenarios in order to identify policy options and effectiveness of different measures for mitigating climate change.…”

“…Nevertheless, such efforts are not entirely convincing, and judging the credibility of climate model projections remains a notorious and unresolved issue (cf. Otto et al 2015).…”

Equipped to deal with uncertainty in climate and impacts predictions: lessons from internal peer review

“…The four papers in this special issue that present quantifications of different elements of the cascade of uncertainty were assessed in detail (Hanlon et al 2013;Otto et al 2003a;Saux Picart et al 2013;Watson et al 2014). The remaining four papers aim to inform uncertainty assessments by examining methodologies for assessment Lorenz et al 2013;Otto et al 2013b;Calel et al 2013). These papers have no quantitative results, so only a limited number of questions in the questionnaires were applicable.…”

“…Two of the three papers not discussed so far (Otto et al 2013b;Allen et al 2013) focus on the development of such generic methods. Allen et al (2013) review a number of approaches to exploring uncertainty in climate model simulations and suggest a simple approach as a first step to making more explicit which approach authors are taking to make studies more comparable.…”

“…All reviewers agree with these authors that it would be beneficial to have an agreed classification system to categorise uncertainties so papers would be more comparable, with one reviewer suggesting that this raises the question whether ensembles should be designed in a standardized way too to make the results of different research more comparable. Otto et al (2013b) propose a new bounding methodology that helps to make and justify judgement on credibility of climate projections based on past performance of substantially dissimilar models with similar difficulty. The reviewers agree that this is a proposal worth exploring, but they also point out the problems with, and judgment required for, determining whether one task is similar in difficulty to another one.…”