ABC for Climate: Dealing with Expensive Simulators

Holden, Philip; Edwards, Neil R.; Hensman, James; Wilkinson, Richard

doi:10.1201/9781315117195-19

Cited by 19 publications

(8 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Approaches such as approximate Bayesian computation (ABC) [68] and history-matching [69,70] change the focus from learning a statistical model within a Bayesian setting, to instead only requiring that the simulation gets within a certain distance of the data. This change, from a fully specified statistical model for δ to instead only giving an upper bound for δ, is a conservative inferential approach where the aim is not to find the best parameter values, but instead rule out only obviously implausible values [71,72].…”

Section: (B) Open Questions and Future Workmentioning

confidence: 99%

Considering discrepancy when calibrating a mechanistic electrophysiology model

Lei

Ghosh

Whittaker

et al. 2020

Phil. Trans. R. Soc. A.

Self Cite

View full text Add to dashboard Cite

Uncertainty quantification (UQ) is a vital step in using mathematical models and simulations to take decisions. The field of cardiac simulation has begun to explore and adopt UQ methods to characterize uncertainty in model inputs and how that propagates through to outputs or predictions; examples of this can be seen in the papers of this issue. In this review and perspective piece, we draw attention to an important and under-addressed source of uncertainty in our predictions—that of uncertainty in the model structure or the equations themselves. The difference between imperfect models and reality is termed model discrepancy , and we are often uncertain as to the size and consequences of this discrepancy. Here, we provide two examples of the consequences of discrepancy when calibrating models at the ion channel and action potential scales. Furthermore, we attempt to account for this discrepancy when calibrating and validating an ion channel model using different methods, based on modelling the discrepancy using Gaussian processes and autoregressive-moving-average models, then highlight the advantages and shortcomings of each approach. Finally, suggestions and lines of enquiry for future work are provided. This article is part of the theme issue ‘Uncertainty quantification in cardiac and cardiovascular modelling and simulation’.

show abstract

Section: (B) Open Questions and Future Workmentioning

confidence: 99%

Considering discrepancy when calibrating a mechanistic electrophysiology model

Lei

Ghosh

Whittaker

et al. 2020

Phil. Trans. R. Soc. A.

Self Cite

View full text Add to dashboard Cite

show abstract

“…These surrogates have been used for many years to allow efficient exploration of the sensitivity of model output to its inputs (L. A. Ryan et al, 2018), generation of large ensembles of model realizations (Holden et al, 2014(Holden et al, , 2019Williamson et al, 2013), and calibration of models (Holden et al, 2015a;Cleary et al, 2021;Couvreux et al, 2021). Although relatively common, these workflows invariably use custom emulators and bespoke analysis routines, limiting their reproducibility and use by non-statisticians.…”

Section: Introductionmentioning

confidence: 99%

Model calibration using ESEm v1.1.0 – an open, scalable Earth system emulator

et al. 2021

View full text Add to dashboard Cite

Abstract. Large computer models are ubiquitous in the Earth sciences. These models often have tens or hundreds of tuneable parameters and can take thousands of core hours to run to completion while generating terabytes of output. It is becoming common practice to develop emulators as fast approximations, or surrogates, of these models in order to explore the relationships between these inputs and outputs, understand uncertainties, and generate large ensembles datasets. While the purpose of these surrogates may differ, their development is often very similar. Here we introduce ESEm: an open-source tool providing a general workflow for emulating and validating a wide variety of models and outputs. It includes efficient routines for sampling these emulators for the purpose of uncertainty quantification and model calibration. It is built on well-established, high-performance libraries to ensure robustness, extensibility and scalability. We demonstrate the flexibility of ESEm through three case studies using ESEm to reduce parametric uncertainty in a general circulation model and explore precipitation sensitivity in a cloud-resolving model and scenario uncertainty in the CMIP6 multi-model ensemble.

show abstract

“…While this added complexity can bring new insights and improved accuracy, sometimes it can be useful to run fast approximations of these models, often referred to as surrogates (Sacks et al, 1989). These surrogates have been used for many years to allow efficient exploration of the sensitivity of model output to its inputs (Lee et al, 2011b;Ryan et al, 2018), generation of large ensembles of model realisations (Holden et al, 2014(Holden et al, , 2019Williamson et al, 2013), and also model calibration (Holden et al, 2015a;Cleary et al, 2021) . Although relatively common, these workflows invariably use custom emulators and bespoke analysis routines, limiting their reproducibility, and use by non-statisticians.…”

Section: Introductionmentioning

confidence: 99%

Model calibration using ESEm v1.0.0 – an open, scalable Earth System Emulator

Watson-Parris

Williams

Deaconu

et al. 2021

Preprint

View full text Add to dashboard Cite

Abstract. Large computer models are ubiquitous in the earth sciences. These models often have tens or hundreds of tuneable parameters and can take thousands of core-hours to run to completion while generating terabytes of output. It is becoming common practice to develop emulators as fast approximations, or surrogates, of these models in order to explore the relationships between these inputs and outputs, understand uncertainties and generate large ensembles datasets. While the purpose of these surrogates may differ, their development is often very similar. Here we introduce ESEm: an open-source tool providing a general workflow for emulating and validating a wide variety of models and outputs. It includes efficient routines for sampling these emulators for the purpose of uncertainty quantification and model calibration. It is built on well-established, high-performance libraries to ensure robustness, extensibility and scalability. We demonstrate the flexibility of ESEm through three case-studies using ESEm to reduce parametric uncertainty in a general circulation model, explore precipitation sensitivity in a cloud resolving model and scenario uncertainty in the CMIP6 multi-model ensemble.

show abstract

ABC for Climate: Dealing with Expensive Simulators

Cited by 19 publications

References 57 publications

Considering discrepancy when calibrating a mechanistic electrophysiology model

Considering discrepancy when calibrating a mechanistic electrophysiology model

Model calibration using ESEm v1.1.0 – an open, scalable Earth system emulator

Model calibration using ESEm v1.0.0 – an open, scalable Earth System Emulator

Contact Info

Product

Resources

About