Low-Rank Independence Samplers in Hierarchical Bayesian Inverse Problems

Abstract

In Bayesian inverse problems, the posterior distribution is used to quantify uncertainty about the reconstructed solution. In practice, Markov chain Monte Carlo algorithms often are used to draw samples from the posterior distribution. However, implementations of such algorithms can be computationally expensive. We present a computationally efficient scheme for sampling high-dimensional Gaussian distributions in ill-posed Bayesian linear inverse problems. Our approach uses Metropolis-Hastings independence sampling with a proposal distribution based on a low-rank approximation of the prior-preconditioned Hessian. We show the dependence of the acceptance rate on the number of eigenvalues retained and discuss conditions under which the acceptance rate is high. We demonstrate our proposed sampler by using it with Metropolis-Hastings-within-Gibbs sampling in numerical experiments in image deblurring, computerized tomography, and NMR relaxometry.

Efficient Marginalization-Based MCMC Methods for Hierarchical Bayesian Inverse Problems

“…An expression for the moments of w(x, θ) can be computed analytically by using properties of Gaussian integrals and was established in [8]. For positive integers m,…”

“…Note that eqs. (8) and (9) are Gamma densities, while eq. (10) is the density of a Gaussian distribution.…”

“…algorithm 1 follows immediately from eqs. (8) to (10) and is precisely the hierarchical Gibbs sampling algorithm of [3].…”

“…To address the second computational issue, i.e., that the cost of computing samples from π(x | b, θ) increases with N , we implement the approach taken in [8], where a low-rank approximation of the so-called prior preconditioned data misfit part of the Hessian is used. This low-rank representation allows efficient sampling from the conditional distribution, reducing the overall computational cost.…”

“…The latter problem can be mitigated using marginalization-based techniques, such as those found in [22,33,45], but these can be computationally prohibitive as well. In this paper, we combine the low-rank techniques of [8] with the marginalization approach of [45]. We consider two variants of this approach: delayed acceptance and pseudo-marginalization.…”