Calibrate, emulate, sample

Abstract

Many parameter estimation problems arising in applications are best cast in the framework of Bayesian inversion. This allows not only for an estimate of the parameters, but also for the quantification of uncertainties in the estimates. Often in such problems the parameterto-data map is very expensive to evaluate, and computing derivatives of the map, or derivative-adjoints, may not be feasible. Additionally, in many applications only noisy evaluations of the map may be available. We propose an approach to Bayesian inversion in such settings that builds on the derivative-free optimization capabilities of ensemble Kalman inversion methods. The overarching approach is to first use ensemble Kalman sampling (EKS) to calibrate the unknown parameters to fit the data; second, to use the output of the EKS to emulate the parameter-to-data map; third, to sample from an approximate Bayesian posterior distribution in which the parameter-to-data map is replaced by its emulator. This results in a principled approach to approximate Bayesian inference that requires only a small number of evaluations of the (possibly noisy approximation of the) parameter-to-data map. It does not require derivatives of this map, but instead leverages the documented power of ensemble Kalman methods. Furthermore, the EKS has the desirable property that it evolves the parameter ensembles towards the regions in which the bulk of the parameter posterior mass is located, thereby locating them well for the emulation phase of the methodology. In essence, the EKS methodology provides a cheap solution to the design problem of where to place points in parameter space to efficiently train an emulator of the parameter-to-data map for the purposes of Bayesian inversion.

A Survey of Active Learning for Quantifying Vegetation Traits from Terrestrial Earth Observation Data

“…Several research areas have addressed the same problem (i.e., AL for regression/emulation) under different names, such as optimal experimental design [59][60][61][62], optimal sensor placements [63,64], generation of quasi-random uniform sequences (Latin Hypercube sampling), Sobol sequences [65,66] and determinantal point processes [67,68], non-uniform, adaptive sampling and quantization of a signal [69,70]. Moreover, adaptive quadrature rules [71,72] and approximations of posterior densities have been introduced [73,74]. Hereby, the notation of AF, explicitly or implicitly defined, is the key point of all these techniques.…”

Statistically Steady State Large‐Eddy Simulations Forced by an Idealized GCM: 1. Forcing Framework and Simulation Characteristics

“…The mismatches are indicative of inadequacies in the GCM's convection parameterization, which is unsurprising given the simplicity of the parameterization. The mismatches can be used to systematically improve parameterization schemes, for example, with Bayesian calibration methods (Cleary et al, ; Schneider et al, ).…”

Uncertainty Quantification of Ocean Parameterizations: Application to the K‐Profile‐Parameterization for Penetrative Convection

“…Such models bring internal sources of uncertainty, and it is not clear to what extent one can trust a surrogate of a full ESM. One potential way to address this additional challenge is the Calibrate, Emulate, and Sample (CES) approach outlined in Cleary et al (2020). There the surrogate model's uncertainty is estimated through the use of Gaussian processes and included as part of a consistent Bayesian calibration procedure.…”