MCMC-Based Image Reconstruction with Uncertainty Quantification

Bardsley, Johnathan M.

doi:10.1137/11085760x

Cited by 83 publications

(154 citation statements)

References 24 publications

Supporting

Mentioning

152

Contrasting

Order By: Relevance

“…However, for more complicated structures, the problem remains critical, especially when H ΛH and G x cannot be diagonalized in the same basis. Other recently proposed algorithms for sampling Gaussian distributions in high dimension follow a two-step perturbation-optimization approach [24,[29][30][31][32][33], which can be summarized as follows:…”

Section: Sampling From High-dimensional Gaussian Distributionmentioning

confidence: 99%

See 1 more Smart Citation

An Auxiliary Variable Method for Markov Chain Monte Carlo Algorithms in High Dimension

Marnissi

Chouzenoux

Benazza-Benyahia

et al. 2018

Entropy

View full text Add to dashboard Cite

Abstract:In this paper, we are interested in Bayesian inverse problems where either the data fidelity term or the prior distribution is Gaussian or driven from a hierarchical Gaussian model. Generally, Markov chain Monte Carlo (MCMC) algorithms allow us to generate sets of samples that are employed to infer some relevant parameters of the underlying distributions. However, when the parameter space is high-dimensional, the performance of stochastic sampling algorithms is very sensitive to existing dependencies between parameters. In particular, this problem arises when one aims to sample from a high-dimensional Gaussian distribution whose covariance matrix does not present a simple structure. Another challenge is the design of Metropolis-Hastings proposals that make use of information about the local geometry of the target density in order to speed up the convergence and improve mixing properties in the parameter space, while not being too computationally expensive. These two contexts are mainly related to the presence of two heterogeneous sources of dependencies stemming either from the prior or the likelihood in the sense that the related covariance matrices cannot be diagonalized in the same basis. In this work, we address these two issues. Our contribution consists of adding auxiliary variables to the model in order to dissociate the two sources of dependencies. In the new augmented space, only one source of correlation remains directly related to the target parameters, the other sources of correlations being captured by the auxiliary variables. Experiments are conducted on two practical image restoration problems-namely the recovery of multichannel blurred images embedded in Gaussian noise and the recovery of signal corrupted by a mixed Gaussian noise. Experimental results indicate that adding the proposed auxiliary variables makes the sampling problem simpler since the new conditional distribution no longer contains highly heterogeneous correlations. Thus, the computational cost of each iteration of the Gibbs sampler is significantly reduced while ensuring good mixing properties.

show abstract

Section: Sampling From High-dimensional Gaussian Distributionmentioning

confidence: 99%

“…In this second experiment, we consider the observation problem defined in (29), where H corresponds to a spatially invariant blur with periodic boundary conditions and the noise is a two-term mixed Gaussian variable; i.e., for every i ∈ {1, . .…”

Section: Problem Formulationmentioning

confidence: 99%

An Auxiliary Variable Method for Markov Chain Monte Carlo Algorithms in High Dimension

Marnissi

Chouzenoux

Benazza-Benyahia

et al. 2018

Entropy

View full text Add to dashboard Cite

show abstract

“…This optimization-based approach for sampling from the posterior density function is implemented in the case of large-scale linear inverse problems in [3]. Also in that paper, the case in which the parameters λ and γ are unknown is treated.…”

Section: P(γ|v λ δ) ∝ P(v|γ λ)P(γ|δ)mentioning

confidence: 99%

“…As is done in [3] and was mentioned above, one can also make the assumption that λ and γ are gamma distributed in the nonlinear case, allowing for γ, λ, and δ to all be sampled from the full posterior p(γ, λ, δ|v) ∝ p(γ|v, λ, δ)p(λ)p(δ). Since RTO requires the Metropolis correction, the resulting sampling scheme has Metropolis-within-Gibbs form.…”

Section: P(γ|v λ δ) ∝ P(v|γ λ)P(γ|δ)mentioning

confidence: 99%

Randomize-Then-Optimize for Sampling and Uncertainty Quantification in Electrical Impedance Tomography

Bardsley¹,

Seppänen²,

Solonen³

et al. 2015

SIAM/ASA J. Uncertainty Quantification

Self Cite

View full text Add to dashboard Cite

Abstract. In a typical inverse problem, a spatially distributed parameter in a physical model is estimated from measurements of model output. Since measurements are stochastic in nature, so is any parameter estimate. Moreover, in the Bayesian setting, the choice of regularization corresponds to the definition of the prior probability density function, which in turn is an uncertainty model for the unknown parameters. For both of these reasons, significant uncertainties exist in the solution of an inverse problem. Thus to fully understand the solution, quantifying these uncertainties is important. When the physical model is linear and the error model and prior are Gaussian, the posterior density function is Gaussian with a known mean and covariance matrix. However, the electrical impedance tomography inverse problem is nonlinear, and hence no closed form expression exists for the posterior density. The typical approach for such problems is to sample from the posterior and then use the samples to compute statistics (such as the mean and variance) of the unknown parameters. Sampling methods for electrical impedance tomography have been studied by various authors in the inverse problems community. However, up to this point the focus has been on the development of increasingly sophisticated implementations of the Gibbs sampler, whose samples are known to converge very slowly to the correct density for large-scale problems. In this paper, we implement a recently developed sampling method called randomize-then-optimize (RTO), which provides nearly independent samples for each application of an appropriate numerical optimization algorithm. The sample density for RTO is not the posterior density, but RTO can be used as a very effective proposal within a Metropolis-Hastings algorithm to obtain samples from the posterior. Here our focus is on implementing the method on synthetic examples from electrical impedance tomography, and we show that it is both computationally efficient and provides good results. We also compare RTO performance with the Metropolis adjusted Langevin algorithm and find RTO to be much more efficient. 1. Introduction. Solving an inverse problem requires estimating unknown parameters in a physical model from measurements of model output. The unknown parameter vector typically corresponds to a numerical discretization of a continuously defined, spatially distributed parameter in the physical model and hence is high-dimensional. This leads to an overparameterized statistical model, and hence estimates have large variance or, in other words, are unstable with respect to perturbations in the observations. Regularization methods [36], which have been studied for decades in the inverse problems literature, are the typical fix for such instability.Over the past ten years or so, an increasing number of researchers in inverse problems have drawn the connection between regularized solutions of inverse problems and Bayesian statistics [20]. Since the measured data in an inverse problem will always contain noise, a ...

show abstract

“…For example, the variance is often estimated from repeated measurements or residual analysis [5], or it can be estimated using a hierarchical model [1,6]. In a number of cases, the Bayesian posterior density function is of least squares form as well.…”

Section: Introductionmentioning

confidence: 99%

Randomize-Then-Optimize: A Method for Sampling from Posterior Distributions in Nonlinear Inverse Problems

Bardsley¹,

Solonen²,

Haario³

et al. 2014

SIAM J. Sci. Comput.

Self Cite

119

165

View full text Add to dashboard Cite

Abstract. High-dimensional inverse problems present a challenge for Markov chain Monte Carlo (MCMC)-type sampling schemes. Typically, they rely on finding an efficient proposal distribution, which can be difficult for large-scale problems, even with adaptive approaches. Moreover, the autocorrelations of the samples typically increase with dimension, which leads to the need for long sample chains. We present an alternative method for sampling from posterior distributions in nonlinear inverse problems, when the measurement error and prior are both Gaussian. The approach computes a candidate sample by solving a stochastic optimization problem. In the linear case, these samples are directly from the posterior density, but this is not so in the nonlinear case. We derive the form of the sample density in the nonlinear case, and then show how to use it within both a Metropolis-Hastings and importance sampling framework to obtain samples from the posterior distribution of the parameters. We demonstrate, with various small-and medium-scale problems, that randomize-then-optimize can be efficient compared to standard adaptive MCMC algorithms.

show abstract

MCMC-Based Image Reconstruction with Uncertainty Quantification

Cited by 83 publications

References 24 publications

An Auxiliary Variable Method for Markov Chain Monte Carlo Algorithms in High Dimension

An Auxiliary Variable Method for Markov Chain Monte Carlo Algorithms in High Dimension

Randomize-Then-Optimize for Sampling and Uncertainty Quantification in Electrical Impedance Tomography

Randomize-Then-Optimize: A Method for Sampling from Posterior Distributions in Nonlinear Inverse Problems

Contact Info

Product

Resources

About