hIPPYlib-MUQ: A Bayesian Inference Software Framework for Integration of Data with Complex Predictive Models under Uncertainty

Abstract

Bayesian inference provides a systematic framework for integration of data with mathematical models to quantify the uncertainty in the solution of the inverse problem. However, solution of Bayesian inverse problems governed by complex forward models described by partial differential equations (PDEs) remains prohibitive with black-box Markov chain Monte Carlo (MCMC) methods. We present hIPPYlib-MUQ, an extensible and scalable software framework that contains implementations of state-of-the art algorithms aimed to overcome the challenges of high-dimensional, PDE-constrained Bayesian inverse problems. These algorithms accelerate MCMC sampling by exploiting the geometry and intrinsic low-dimensionality of parameter space via derivative information and low rank approximation. The software integrates two complementary opensource software packages, hIPPYlib and MUQ. hIPPYlib solves PDE-constrained inverse problems using automatically-generated adjoint-based derivatives, but it lacks full Bayesian capabilities. MUQ provides a spectrum of powerful Bayesian inversion models and algorithms, but expects forward models to come equipped with gradients and Hessians to permit large-scale solution. By combining these two complementary libraries, we created a robust, scalable, and efficient software framework that realizes the benefits of each and allows us to tackle complex large-scale Bayesian inverse problems across a broad spectrum of scientific and engineering disciplines. To illustrate the capabilities of hIPPYlib-MUQ, we present a comparison of a number of MCMC methods available in the integrated software on several high-dimensional Bayesian inverse problems. These include problems characterized by both linear and nonlinear PDEs, low and high levels of data noise, and different parameter dimensions. The results demonstrate that large (∼ 50×) speedups over conventional black box and gradient-based MCMC algorithms can be obtained by exploiting Hessian information (from the log-posterior), underscoring the power of the integrated hIPPYlib-MUQ framework.

Hierarchical off-diagonal Hessian approximation for Bayesian inverse problems with application to the flow of the Greenland ice sheet.

“…Another MCMC sampling approach is the generalized preconditioned Crank-Nicholson (gpCN) method [27,28]. An attractive choice for the preconditioner is the Hessian at the MAP point, [29]. For these and other MCMC samplers, one typically needs to apply the inverse Hessian H −1 k or its square root H −1/2 k repeatedly and efficiently, which also motivates the study presented in this paper.…”

Efficient algorithms for Bayesian Inverse Problems with Whittle--Matérn Priors

“…This choice has two advantages: first, it is integer-valued, and second, it is easy to obtain a factored form of the covariance operator. For these reasons, this approach is attractive from a computational standpoint, and has since become popular and has resulted in scalable software implementations [31,21]. A similar approach can also be found in [25], in which the authors also used the Whittle-Matérn priors for Bayesian inverse problems.…”

A gradient-free subspace-adjusting ensemble sampler for infinite-dimensional Bayesian inverse problems

“…Other samplers with dimension-independent convergence properties, utilizing derivative information, include ∞-MALA, ∞-HMC and their manifold variants [2] and DILI [7]. The papers [2,20] provide a systematic comparison of a number of the above algorithms applied to high-dimensional Bayesian inverse problems. Outside of MCMC, [1] considers the performance of importance sampling on general state spaces, and its dependence on the discretization dimension and effective dimension of the problem.…”