A variational Bayesian method to inverse problems with impulsive noise

Abstract

We propose a novel numerical method for solving inverse problems subject to impulsive noises which possibly contain a large number of outliers. The approach is of Bayesian type, and it exploits a heavy-tailed t distribution for data noise to achieve robustness with respect to outliers. A hierarchical model with all hyper-parameters automatically determined from the given data is described.An algorithm of variational type by minimizing the Kullback-Leibler divergence between the true posteriori distribution and a separable approximation is developed. The numerical method is illustrated on several one-and two-dimensional linear and nonlinear inverse problems arising from heat conduction, including estimating boundary temperature, heat flux and heat transfer coefficient. The results show its robustness to outliers and the fast and steady convergence of the algorithm. key words: impulsive noise, robust Bayesian, variational method, inverse problems IntroductionWe are interested in Bayesian approaches for inverse problems subject to impulsive noises. Bayesian inference provides a principled framework for solving diverse inverse problems, and has demonstrated distinct features over deterministic techniques, e.g., Tikhonov regularization. Firstly, it can yield an ensemble of plausible solutions consistent with the given data. This enables quantifying the uncertainty of a specific solution, e.g., with credible intervals. In contrast, deterministic techniques generally content with singling out one solution from the ensemble. Secondly, it provides a flexible regularization since hierarchical modeling can partially resolve the nontrivial issue of choosing an appropriate regularization parameter.It is known that the underlying mechanism is balancing principle [22]. Thirdly, it allows seamlessly integrating structural/multiscale features of the problem through careful prior modeling. Therefore, it has attracted attention in a wide variety of applied disciplines, e.g., geophysics [37,34], medical imaging * [18, 27, 1] and heat conduction [13,39, 40,14], see also [32,28,30,31] for other applications. For an overview of methodological developments, we refer to the monographs [37,26].Amongst existing studies on Bayesian inference for inverse problems, the Gaussian noise model has played a predominant role. This is often justified by appealing to central limit theorem. The theorem asserts that the normal distribution is a suitable model for data that are formed as the sum of a large number of independent components. Even in the absence of such justifications, this model is still preferred due to its computational/analytical conveniences, i.e., it allows direct computation of the posterior mean and variance and easy exploration of the posterior state space (for linear models with Gaussian priors). A well acknowledged limitation of the Gaussian model is its lack of robustness against the outliers, i.e., data points that lie far away from the bulk of the data, in the observations: A single aberrant data point can significantly influ...

A variational Bayesian approach for inverse problems with skew-t error distributions

“…The present work extends prior work [17] in two aspects. First, this work considers the skew-t distribution for the skewness of data errors, whereas [17] considers only the t-distribution.…”

“…In this paper we investigate an alternative approach based on the variational method [20,19,24]. In spite of its wide popularity in the machine learning community, the application of variational methods to inverse problems seems largely unexplored [23,18,17,13]. Tipping and Lawrence (2005) [23] and Jin (2012) [17] developed Bayesian approaches to inverse problems with a heavytailed t model to cope data with outliers.…”

“…In spite of its wide popularity in the machine learning community, the application of variational methods to inverse problems seems largely unexplored [23,18,17,13]. Tipping and Lawrence (2005) [23] and Jin (2012) [17] developed Bayesian approaches to inverse problems with a heavytailed t model to cope data with outliers. Our proposed approach generalizes the method developed in [17] by a robust Bayesian formulation of the inverse problem using the skew-t distribution and a sparse prior structure.…”

“…Tipping and Lawrence (2005) [23] and Jin (2012) [17] developed Bayesian approaches to inverse problems with a heavytailed t model to cope data with outliers. Our proposed approach generalizes the method developed in [17] by a robust Bayesian formulation of the inverse problem using the skew-t distribution and a sparse prior structure. The attractive features of this approach are (i) uncertainty quantification of the computed solution, (ii) robustness to data outliers, and (iii) general applicability to both linear and non-linear inverse problems.…”

“…First, this work considers the skew-t distribution for the skewness of data errors, whereas [17] considers only the t-distribution. The skewness in the error distribution introduces an extra layer of the computational complexity in developing efficient inference algorithms.…”