On the evaluation of climate change impact models

Abstract

In-depth understanding of the potential implications of climate change is required to guide decision-and policy-makers when developing adaptation strategies and designing infrastructure suitable for future conditions. Impact models that translate potential future climate conditions into variables of interest are needed to create the causal connection between a changing climate and its impact for different sectors. Recent surveys suggest that the primary strategy for validating such models (and hence for justifying their use) heavily relies on assessing the accuracy of model simulations by comparing them against historical observations. We argue that such a comparison is necessary and valuable, but not sufficient to achieve a comprehensive evaluation of climate change impact models. We believe that a complementary, largely observation-independent, step of model evaluation is needed to ensure more transparency of model behavior and greater robustness of scenario-based analyses. This step should address the following four questions: (1) Do modeled dominant process controls match our system perception? (2) Is my model's sensitivity to changing forcing as expected? (3) Do modeled decision levers show adequate influence? (4) Can we attribute uncertainty sources throughout the projection horizon? We believe that global sensitivity analysis, with its ability to investigate a model's response to joint variations of multiple inputs in a structured way, offers a coherent approach to address all four questions comprehensively. Such additional model evaluation would strengthen stakeholder confidence in model projections and, therefore, into the adaptation strategies derived with the help of impact models. This article is categorized under: • Climate Models and Modeling > Knowledge Generation with Models • Assessing Impacts of Climate Change > Evaluating Future Impacts of Climate Change

Uncertainty and sensitivity analysis for probabilistic weather and climate-risk modelling: an implementation in CLIMADA v.3.1.0

“…In practice, the quantification of risk with climate-risk models is particularly challenging as it involves dealing with the absence of robust verification data (Matott et al, 2009;Wagener et al, 2022) when setting up the hazard, exposure, and vulnerability sub-models, as well as dealing with large uncertainties in the input parameters and the model structure itself (Knüsel, 2020). For example, in hazard modelling, many authors have shown large uncertainties affecting the computation of flood maps through hydraulic modelling (Merwade et al, 2008;Dottori et al, 2013); similarly, alternative models have been proposed for modelling tracks and intensities of tropical cyclones (Emanuel, 2017;Bloemendaal et al, 2020).…”

Capacity at risk: a metric for robust planning decisions under uncertainty in the electric sector

“…Existing metrics such as the expected value of information and the value of the stochastic solution typically assume the availability of a stochastic model (Birge 1982, Bistline 2015. There are sensitivity frameworks based on deterministic model runs to guide decision-makers, analysts, and other stakeholders on whether more sophisticated methods are appropriate and to prioritize information gathering on uncertain factors (Herman et al 2015, Pianosi et al 2016, Quinn et al 2019, Wagener et al 2022. However, these approaches have largely been neglected in energy systems context in favor of stochastic optimization approaches.…”

Addressing challenges in uncertainty quantification. The case of geohazard assessments

“…Surrogate models are learnt in order to replace a complicated model with an inexpensive and fast approximation. Parameters reduction is achieved based on either principal component analysis or global sensitivity analysis to determine which parameters significantly impact model outputs and are essential to the 160 analysis (Degen et al, 2022;Wagener, Reinecke, and Pianosi, 2022). Remarkably, versions of the model learning option do not need any prior information about model equations Eɱ but require local verification of conservation laws in the data ɗ (Lu and Lermusiaux, 2021).…”

“…O1. Credibility of predictions is judged in terms of physical consistency checks (Wagener, Reinecke, and Pianosi, 2022) and by examining the ability of models to reproduce disruptive changes recorded in the data (Alley, 2004). O2.…”

“…Recall that model accuracy usually relates to the comparison of model outputs with experimental measurements(Roy and Oberkampf, 2011;Aven and Zio, 2013) and is the basis for validating models. Regarding credibility of predictions,Wagener, Reinecke, and Pianosi (2022) have reported that such credibility can be mainly judged in terms of the physical consistencyhttps://doi.org/10.5194/gmd-2022-210 Preprint. Discussion started: 17 October 2022 c Author(s) 2022.…”