Bayesian Antisparse Coding

Abstract

Sparse representations have proven their efficiency in solving a wide class of inverse problems encountered in signal and image processing. Conversely, enforcing the information to be spread uniformly over representation coefficients exhibits relevant properties in various applications such as robust encoding in digital communications. Anti-sparse regularization can be naturally expressed through an ∞-norm penalty. This paper derives a probabilistic formulation of such representations. A new probability distribution, referred to as the democratic prior, is first introduced. Its main properties as well as three random variate generators for this distribution are derived. Then this probability distribution is used as a prior to promote anti-sparsity in a Gaussian linear model, yielding a fully Bayesian formulation of anti-sparse coding. Two Markov chain Monte Carlo (MCMC) algorithms are proposed to generate samples according to the posterior distribution. The first one is a standard Gibbs sampler. The second one uses Metropolis-Hastings moves that exploit the proximity mapping of the log-posterior distribution. These samples are used to approximate maximum a posteriori and minimum mean square error estimators of both parameters and hyperparameters. Simulations on synthetic data illustrate the performances of the two proposed samplers, for both complete and over-complete dictionaries. All results are compared to the recent deterministic variational FITRA algorithm.

Split-and-Augmented Gibbs Sampler—Application to Large-Scale Inference Problems

“…Capitalizing on the advantages of proximal splitting recently popularized to solve large-scale inference problems [13]- [18], the proximal Monte Carlo method allows high-dimensional log-concave distributions to be sampled. For instance, this algorithm has been successfully used to conduct antisparse coding [19] and has been significantly improved in [20].…”

Alternative Criteria for Predictive Blind Deconvolution

“…• For sub Gaussian distributions [21], we adopt p > 2, with p → ∞ for uniform distribution. From these results we observe two dual potential scenarios for predictive deconvolution: 1) the case where the desired signal is super Gaussian (sparsity property to be explored) and an p predictor may be used with 1 ≤ p ≤ 2 and 2) the case where the desired signal is sub Gaussian (antisparsity property [11] to be explored) and it is suitable to use p > 2.…”

“…where λ(n) is a weight factor that controls the degree of relevance of the error sample at time instant n, e(n) is the error signal and w are the filter coefficients. Considering stationary and ergodic processes [18], the criteria given by (4) and (11) are equivalent for practical purposes. So, minimizing the MSE criterion is equivalent to minimize the least squares (LS) criterion in (11).…”

“…The present paper extends the study in [10], investigating the zero location and phase response of the p PEF, and presents several new results. Such results concern specially a deeper assessment about the zero location (and then, the phase response) of the p PEF, when the value of p is adjusted according to the properties of sparsity and antisparsity [11] of the signal to be recovered. All the results have been obtained in a noise free scenario with real valued signals.…”

Cascade of Linear Predictors for Deconvolution of Non-Stationary Channels in Sparse and Antisparse Scenarios

“…where 2 ( ) = ( − ) − =1 ( − + 1). One of our interests in this work is to perform unsupervised deconvolution in order to recover telecommunication signals, which usually follow a uniform distribution [14], and, hence, present an antisparse structure [15]. Therefore, as we discussed in [7], the most suitable measure for this property is the ℓ ∞ norm, which is equivalent to the Maximum Likelihood estimator for a uniform distribution.…”