Derivative-Free Bayesian Inversion Using Multiscale Dynamics

Abstract

Inverse problems are ubiquitous because they formalize the integration of data with mathematical models. In many scientific applications the forward model is expensive to evaluate, and adjoint computations are difficult to employ; in this setting derivative-free methods which involve a small number of forward model evaluations are an attractive proposition. Ensemble Kalman based interacting particle systems (and variants such as consensus based and unscented Kalman approaches) have proven empirically successful in this context, but suffer from the fact that they cannot be systematically refined to return the true solution, except in the setting of linear forward models [20]. In this paper, we propose a new derivative-free approach to Bayesian inversion, which may be employed for posterior sampling or for maximum a posteriori (MAP) estimation, and may be systematically refined. The method relies on a fast/slow system of stochastic differential equations (SDEs) for the local approximation of the gradient of the log-likelihood appearing in a Langevin diffusion. Furthermore the method may be preconditioned by use of information from ensemble Kalman based methods (and variants), providing a methodology which leverages the documented advantages of those methods, whilst also being provably refineable. We define the methodology, highlighting its flexibility and many variants, provide a theoretical analysis of the proposed approach, and demonstrate its efficacy by means of numerical experiments.

Propagation of chaos: A review of models, methods and applications. Ⅱ. Applications

“…The present method as well as various other EKI methods are investigated numerically in [262]. A new methodology for nonlinear settings can be found in [257].…”

Consensus Based Sampling

“…Like the ensemble Kalman sampler, the CBS approach is only exact for Gaussian problems and in the mean field limit. However recently developed methods based on multiscale stochastic dynamics provide a refineable methodology for sampling from non-Gaussian distributions [52]; methods such as CBS or EKS may be used to precondition these multiscale stochastic dynamics algorithms, making them more efficient. Alternatively, the CBS method may be used in the calibration step employed within the calibrate-emulate-sample methodology introduced in [15].…”

“…In this section, we consider the more challenging inverse problem of finding the permeability field of a porous medium from noisy pressure measurements in a Darcy flow; for other methods applied to this problem, see [17,26,52]. Assuming Dirichlet boundary conditions and scalar permeability for simplicity, we consider the forward model mapping the logarithm of the permeability, denoted by a(x), to the solution of the PDE…”

A gradient-free subspace-adjusting ensemble sampler for infinite-dimensional Bayesian inverse problems

“…Outside of MCMC, [1] considers the performance of importance sampling on general state spaces, and its dependence on the discretization dimension and effective dimension of the problem. Variations of the ensemble Kalman Filter (EnKF) [11] have also been considered in the context of Bayesian inversion on general state spaces [14,22], allowing for derivative-free exploration of the posterior via approximate natural Langevin dynamics [14].…”