MAP estimators and their consistency in Bayesian nonparametric inverse problems

Dashti, Masoumeh; Law, Kody J. H.; Stuart, Andrew M.; Voß, Jochen

doi:10.1088/0266-5611/29/9/095017

Cited by 146 publications

(284 citation statements)

References 37 publications

Supporting

Mentioning

276

Contrasting

Order By: Relevance

“…The resulting framework is appropriate for the mathematical analysis of inverse problems, as well as the development of algorithms. For example, on the analysis side, the idea of MAP estimators, which links the Bayesian approach with classical regularization, developed for Gaussian priors in [30], has recently been extended to other prior models in [47]; the study of contraction of the posterior distribution to a Dirac measure on the truth underlying the data is undertaken in [3,4,99]. On the algorithmic side, algorithms for Bayesian inversion in geophysical applications are formulated in [16,81], and on the computational statistics side, methods for optimal experimental design are formulated in [5,6].…”

Section: Discussionmentioning

confidence: 99%

The Bayesian Approach to Inverse Problems

Dashti

Stuart

2015

Handbook of Uncertainty Quantification

143

222

View full text Add to dashboard Cite

These lecture notes highlight the mathematical and computational structure relating to the formulation of, and development of algorithms for, the Bayesian approach to inverse problems in differential equations. This approach is fundamental in the quantification of uncertainty within applications involving the blending of mathematical models with data. The finite dimensional situation is described first, along with some motivational examples. Then the development of probability measures on separable Banach space is undertaken, using a random series over an infinite set of functions to construct draws; these probability measures are used as priors in the Bayesian approach to inverse problems. Regularity of draws from the priors is studied in the natural Sobolev or Besov spaces implied by the choice of functions in the random series construction, and the Kolmogorov continuity theorem is used to extend regularity considerations to the space of Hölder continuous functions. Bayes' theorem is derived in this prior setting, and here interpreted as finding conditions under which the posterior is absolutely continuous with respect to the prior, and determining a formula for the Radon-Nikodym derivative in terms of the likelihood of the data. Having established the form of the posterior, we then describe various properties common to it in the infinite dimensional setting. These properties include well-posedness, approximation theory, and the existence of maximum a posteriori estimators. We then describe measure-preserving dynamics, again on the infinite dimensional space, including Markov chain-Monte Carlo and sequential Monte Carlo methods, and measure-preserving reversible stochastic differential equations. By formulating the theory and algorithms on the underlying infinite dimensional space, we obtain a framework suitable for rigorous analysis of the accuracy of reconstructions, of computational complexity, as well as naturally constructing algorithms which perform well under mesh refinement, since they are inherently well-defined in infinite dimensions.

show abstract

Section: Discussionmentioning

confidence: 99%

The Bayesian Approach to Inverse Problems

Dashti

Stuart

2015

Handbook of Uncertainty Quantification

143

222

View full text Add to dashboard Cite

show abstract

“…Optimal Control Approach ( [30]. Therefore, variational estimation/DA is a method based on the optimal control theory, which can also be understood as a special case of the maximum a-posteriory probability (MAP) estimator [8]. This method is preferred for weather and ocean forecasting in major operational centers around the globe, particularly in the form of the incremental 4D-Var [7], and in the form of the ensemble 4D-Var [6].…”

Section: Msc 2010: 65k10 86a22 93b40mentioning

confidence: 99%

Hessian-based covariance approximations in variational data assimilation

Gejadze

Shutyaev

Dimet

2018

Russian Journal of Numerical Analysis and Mathematical Modelling

View full text Add to dashboard Cite

Abstract:The problem of variational data assimilation (estimation) for a nonlinear model is considered in general operator formulation. Hessian-based methods are presented to compute the estimation error covariances. The importance of dynamic formulation and the role of the Hessian and its inverse are discussed. Keywords:Variational data assimilation, optimality system, Hessian, estimation error covariance. MSC 2010: 65K10, 86A22, 93B40Data assimilation (state or parameter estimation) for high-dimensional distributed parameter dynamical systems has become a powerful analysis tool in recent decades. It is a fusion process of incomplete and, possibly, indirect observations of state variables with a mathematical model governed by partial differential equations, complemented by a priori information. The applications include model initialization in meteorology and oceanography, air and water quality monitoring, 'calibration' of groundwater and reservoir models, discharge estimation and forecasting in river hydraulics and hydrology, flow estimation and control in aerospace engineering, process control in chemical and nuclear engineering, etc. In different applications these estimation problems are also referred as 'inverse problems', 'data assimilation' (DA), and 'calibration'. Variational methods were introduced in meteorology by Sasaki [27]. These methods consider the equations governing the flow as constraints and the problem is closed by using a variational principle, e.g., the minimization of the discrepancy between the model prediction and the observations. Optimal Control Approach ( [30]. Therefore, variational estimation/DA is a method based on the optimal control theory, which can also be understood as a special case of the maximum a-posteriory probability (MAP) estimator [8]. This method is preferred for weather and ocean forecasting in major operational centers around the globe, particularly in the form of the incremental 4D-Var [7], and in the form of the ensemble 4D-Var [6]. Variational estimation is widely used in other scientific and engineering applications, such as aerospace engineering [2] and astrophysics [4], to mention a few.Uncertainty quantification is an important topic closely associated with estimation and data assimilation. An overview of the original works by the authors on the Hessian-based advanced methodology for computing the estimation error covariances in the framework of variational estimation is presented in the coming sections. The main results are given in a general operator formulation. The accent is made on feasibility of the suggested methods for the uncertainty quantification in high-dimensions, where the statistical methods (Monte Carlo involving associated tricks, e.g., localization, importance sampling, etc.) may not produce a sensible outcome due to a very small sample being available. Some of the approaches could equally be useful for improving (in terms of acceleration, memory savings and robustness) the estimation/DA technique itself.

show abstract

“…One commonly adopted approach to Bayesian inverse problems, which is capable of capturing such multiple modes, is to find the maximum a posteriori (MAP) estimator. This corresponds to identifying the center of balls of maximal probability in the limit of vanishingly small radius [12,20]; this is linked to the classical theory of Tikhonov-Phillips regularization of inverse problems [15]. Another commonly adopted approach is to employ Monte Carlo Markov chain (MCMC) methods [24] to sample the probability measure of interest.…”

Section: Infinite Dimensional Motivationmentioning

confidence: 99%

Kullback--Leibler Approximation for Probability Measures on Infinite Dimensional Spaces

Pinski¹,

Simpson²,

Stuart³

et al. 2015

SIAM J. Math. Anal.

View full text Add to dashboard Cite

Abstract. In a variety of applications it is important to extract information from a probability measure μ on an infinite dimensional space. Examples include the Bayesian approach to inverse problems and (possibly conditioned) continuous time Markov processes. It may then be of interest to find a measure ν, from within a simple class of measures, which approximates μ. This problem is studied in the case where the Kullback-Leibler divergence is employed to measure the quality of the approximation. A calculus of variations viewpoint is adopted, and the particular case where ν is chosen from the set of Gaussian measures is studied in detail. Basic existence and uniqueness theorems are established, together with properties of minimizing sequences. Furthermore, parameterization of the class of Gaussians through the mean and inverse covariance is introduced, the need for regularization is explained, and a regularized minimization is studied in detail. The calculus of variations framework resulting from this work provides the appropriate underpinning for computational algorithms. 1. Introduction. This paper is concerned with the problem of minimizing the Kullback-Leibler divergence between a pair of probability measures, viewed as a problem in the calculus of variations. We are given a measure μ, specified by its RadonNikodym derivative with respect to a reference measure μ 0 , and we find the closest element ν from a simpler set of probability measures. After an initial study of the problem in this abstract context, we specify to the situation where the reference measure μ 0 is Gaussian and the approximating set comprises Gaussians. It is necessarily the case that minimizers ν are then equivalent as measures to μ 0 , 1 and we use the Feldman-Hajek theorem to characterize such ν in terms of their inverse covariance operators. This induces a natural formulation of the problem as minimization over the mean, from the Cameron-Martin space of μ 0 , and over an operator from a weighted Hilbert-Schmidt space. We investigate this problem from the point of view of the calculus of variations, studying properties of minimizing sequences, regularization to improve the space in which operator convergence is obtained, and uniqueness under a slight strengthening of a log-convex assumption on the measure μ.In the situation where the minimization is over a convex set of measures ν, the problem is classical and completely understood [10]; in particular, there is uniqueness

show abstract

MAP estimators and their consistency in Bayesian nonparametric inverse problems

Cited by 146 publications

References 37 publications

The Bayesian Approach to Inverse Problems

The Bayesian Approach to Inverse Problems

Hessian-based covariance approximations in variational data assimilation

Kullback--Leibler Approximation for Probability Measures on Infinite Dimensional Spaces

Contact Info

Product

Resources

About