Overcomplete representation in a hierarchical Bayesian framework

Abstract

A common task in inverse problems and imaging is finding a solution that is sparse, in the sense that most of its components vanish. In the framework of compressed sensing, general results guaranteeing exact recovery have been proven. In practice, sparse solutions are often computed combining 1 -penalized least squares optimization with an appropriate numerical scheme to accomplish the task. A computationally efficient alternative for finding sparse solutions to linear inverse problems is provided by Bayesian hierarchical models, in which the sparsity is encoded by defining a conditionally Gaussian prior model with the prior parameter obeying a generalized gamma distribution. An iterative alternating sequential (IAS) algorithm has been demonstrated to lead to a computationally efficient scheme, and combined with Krylov subspace iterations with an early termination condition, the approach is particularly well suited for large scale problems. Here the Bayesian approach to sparsity is extended to problems whose solution allows a sparse coding in an overcomplete system such as composite frames. It is shown that among the multiple possible representations of the unknown, the IAS algorithm, and in particular, a hybrid version of it, is effectively identifying the most sparse solution. Computed examples show that the method is particularly well suited not only for traditional imaging applications but also for dictionary learning problems in the framework of machine learning.

Bayesian hierarchical dictionary learning

“…The IAS-based unsupervised dictionary learning algorithm with the full training set as dictionary, typically yielding an overcomplete frame to represent the data, has been discussed in detail in [25]. Here we restrict the discussion to the problem of learning an economy-size dictionary.…”

Adaptive anisotropic Bayesian meshing for inverse problems

“…The meshing is a function of an underlying metric that is updated at each iteration concomitantly with the approximate solution. The computational engine used for estimating the sparse representation of the gradient is an iterative numerical scheme based on hierarchical Bayesian hypermodels, a hybrid version of the iterative alternating scheme (IAS) studied extensively in the literature [8,9,13,17,18,31,32].…”