A Measure-Theoretic Variational Bayesian Algorithm for Large Dimensional Problems

Abstract

Abstract. In this paper we provide an algorithm allowing to solve the variational Bayesian issue as a functional optimization problem. The main contribution of this paper is to transpose a classical iterative algorithm of optimization in the metric space of probability densities involved in the Bayesian methodology. The main advantage of this methodology is that it allows to address large dimensional inverse problems by unsupervised algorithms. The interest of our algorithm is enhanced by its application to large dimensional linear inverse problems involving sparse objects. Finally, we provide simulation results. First we show the good numerical performances of our method by comparing it with classical ones on a small tomographic problem. On a second time we treat a large dimensional dictionary learning problem and compare our method with a wavelet based one.

Efficient Variational Bayesian Approximation Method Based on Subspace Optimization

“…As stated in [11], May 12, 2014 DRAFT there exists a problem equivalent to (5) in a separable probability measure space A = P i=1 A i , the Cartesian product of the A i , which is defined as follows:…”

“…In [11], the gradient descent method in Hilbert spaces has been transposed into the space of pdfs and, as a result, an exponentiated gradient based variational Bayesian approximation (EGrad-VBA) method whose convergence is proven was proposed to solve the involved functional optimization problem. For the aim of developing more efficient methods, we transpose here in the same context the subspace optimization method which has been shown to outperform standard optimization methods, such as gradient or conjugate gradient methods, in terms of rate of convergence in finite dimensional Hilbert spaces [17].…”

“…In order to obtain more efficient variational Bayesian approaches, a different method has been recently introduced in [11]. It is based on the adaptation of the exponentiated gradient algorithm [12] into the space of pdf, which is no longer a Hilbert space.…”

“…In order to further improve the method of [11], a natural idea is to consider a new descent direction.…”

“…Bayesian algorithm proposed in [11]. These comparisons are based on an implementation on a superresolution problem [23] which aims to reconstruct a high resolution image from several low resolution ones representing the same scene.…”