Calibrating Climate Model Ensembles for Assessing Extremes in a Changing Climate

Abstract

Climate models serve as indispensable tools to investigate the effect of anthropogenic emissions on current and future climate, including extremes. However, as low‐dimensional approximations of the climate system, they will always exhibit biases. Several attempts have been made to correct for biases as they affect extremes prediction, predominantly focused on correcting model‐simulated distribution shapes. In this study, the effectiveness of a recently published quantile‐based bias correction scheme, as well as a new subset selection method introduced here, are tested out‐of‐sample using model‐as‐truth experiments. Results show that biases in the shape of distributions tend to persist through time, and therefore, correcting for shape bias is useful for past and future statements characterizing the probability of extremes. However, for statements characterized by a ratio of the probabilities of extremes between two periods, we find that correcting for shape bias often provides no skill improvement due to the dominating effect of bias in the long‐term trend. Using a toy model experiment, we examine the relative importance of the shape of the distribution versus its position in response to long‐term changes in radiative forcing. It confirms that the relative position of the two distributions, based on the trend, is at least as important as the shape. We encourage the community to consider all model biases relevant to their metric of interest when using a bias correction procedure and to construct out‐of‐sample tests that mirror the intended application.

Taking climate model evaluation to the next level

“…However, only a few studies have specifically focused on the likelihood of weighted results providing benefits for the intended application (that is, out-of-sample, typically twenty-first century projections) [10][11][12]14,15,68,71,72 . Although we clearly have no observations of future climate, model-as-truth (also termed pseudo-reality in some studies 11,68 ) and calibration-validation exercises for different time periods of the observations yield valuable information on the potential benefits of different weighting approaches 73 . In addition to testing whether projections of a specific variable and metric can be improved through weighting, thorough out-of-sample testing can help guard against other potential issues with weighting.…”

Streamflow-based evaluation of climate model sub-selection methods

“…Therefore, it is not always possible to present all available ensemble members, e.g., when complex model cascades are used (Kiesel et al 2019a) or when multiple other sources of uncertainty are of interest as well (Clark et al 2016). While the practice of sub-selecting large model ensembles has been criticized in the past (Mote et al 2011;Christensen et al 2010), it is now a generally accepted approach (Eyring et al 2019;Herger et al 2018;Knutti et al 2017). This is mainly due to the fact that informed model sampling and reduction of informational redundancy in the ensemble are expected to improve climate change impact assessments (Eyring et al 2019;Pechlivanidis et al 2018).…”

Model dependence in multi-model climate ensembles: weighting, sub-selection and out-of-sample testing

“…Even the somewhat naive approach of institutional democracy is likely to be a less biased approach to ensemble sampling. Nonetheless we discourage the use of weighting or sub-sampling without out-of-sample 25 testing, as the risks may well outweigh the potential benefits (Weigel et al, 2010;Herger et al, 2018b).…”

“…The second approach is model-as-truth, or perfect model experiments (e.g. Abramowitz and Bishop, 2015;Sanderson et al, 2017;Knutti et al, 2017;Herger et al, 2018a, Herger et al, 2018b. This involves removing one of the ensemble members and treating it as though it were observations.…”