Complexity analysis of accelerated MCMC methods for Bayesian inversion

Abstract

The Bayesian approach to inverse problems, in which the posterior probability distribution on an unknown field is sampled for the purposes of computing posterior expectations of quantities of interest, is starting to become computationally feasible for partial differential equation (PDE) inverse problems. Balancing the sources of error arising from finite-dimensional approximation of the unknown field, the PDE forward solution map and the sampling of the probability space under the posterior distribution are essential for the design of efficient computational Bayesian methods for PDE inverse problems. We study Bayesian inversion for a model elliptic PDE with an unknown diffusion coefficient. We provide complexity analyses of several Markov chain Monte Carlo (MCMC) methods for the efficient numerical evaluation of expectations under the Bayesian posterior distribution, given data δ. Particular attention is given to bounds on the overall work required to achieve a prescribed error level ε. Specifically, we first bound the computational complexity of 'plain' MCMC, based on combining MCMC sampling with linear complexity multi-level solvers for elliptic PDE. Our (new) work versus accuracy bounds show that the complexity of this approach can be quite prohibitive. Two strategies for reducing the computational complexity are then proposed and analyzed: first, a sparse, parametric and deterministic generalized polynomial chaos (gpc) 'surrogate' representation of the forward response map of the PDE over the entire parameter space, and, second, a novel multi-level Markov chain Monte Carlo strategy which utilizes sampling from a multi-level discretization of the posterior and the forward PDE. For both of these strategies, we derive asymptotic bounds on work versus accuracy, and hence asymptotic bounds on the computational complexity of the algorithms. In particular, we provide sufficient conditions on the regularity of the unknown coefficients of the PDE and on the approximation methods used, in order for the accelerations

Correction of coarse-graining errors by a two-level method: Application to the Asakura-Oosawa model

“…where we used that the denominator of (20) converges for large N f to the expectation of (19) with respect to p c , which is (Ξ f /Ξ c ). [The approximate equality in (21) is accurate up to a small correction associated with the difference between the denominator of (20) and its average, see Sec.…”

“…A consistent choice of the weight is thenŴ (C) = Ξ S (µ 0 , C)e βI(C)+βUc(C) . Then Crooks' fluctuation formula 27 (analogous to Jarzynski's equality 26 ) can be used to show that (19) holds. A short derivation is given in Appendix A 2, see also Refs.…”

“…which is similar to (11) with µ S replaced by µ k . To establish (19) from (25) it is sufficient to show that e βI(C)…”

“…Our method is inspired by recent mathematical studies, particularly the multilevel approach, [16][17][18] which has been developed recently for applications in Bayesian statistics and uncertainty quantification. [19][20][21] (We note that the term "multi-level" has a specific mathematical meaning and should not be confused with the "multi-scale" terminology used in physics and chemistry. 10,11 ) Our method corresponds to a multi-level implementation with only two levels, so we call it a two-level (TL) method.…”

Sparse-grid, reduced-basis Bayesian inversion

“…Built on the classical Markov chain Monte Carlo (MCMC) method [21], the geometry of the (log) posterior as captured by its gradient, Hessian, and higher order derivatives with respect to (w.r.t.) the parameter has been exploited to accelerate the convergence of MCMC [20,31,23,4,40,24,34,2]. By taking advantage of the smoothness and sparsity of the posterior w.r.t.…”