Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing

Rangan, Sundeep; Fletcher, Alyson K.; Goyal, Vivek K

doi:10.1109/tit.2011.2177575

Cited by 173 publications

(319 citation statements)

References 65 publications

(131 reference statements)

Supporting

Mentioning

300

Contrasting

Order By: Relevance

“…1 is sometimes referred to as a generalized linear model [2], [3] and is used in a range of applications including regression, inverse problems, and filtering. Bayesian forms of compressed sensing can also be considered in this framework by imposing a sparse prior for the components x j [4], [5]. In all these applications, one may instead be interested in estimating the posterior marginals p(x j |y) and p(z i |y).…”

Section: Introductionmentioning

confidence: 99%

Fixed points of generalized approximate message passing with arbitrary matrices

Rangan¹,

Schniter

Riegler

et al. 2013

2013 IEEE International Symposium on Information Theory

Self Cite

View full text Add to dashboard Cite

Abstract-The estimation of a random vector with independent components passed through a linear transform followed by a componentwise (possibly nonlinear) output map arises in a range of applications. Approximate message passing (AMP) methods, based on Gaussian approximations of loopy belief propagation, have recently attracted considerable attention for such problems. For large random transforms, these methods exhibit fast convergence and admit precise analytic characterizations with testable conditions for optimality, even for certain non-convex problem instances. However, the behavior of AMP under general transforms is not fully understood. In this paper, we consider the Generalized AMP (GAMP) algorithm and relate the method to more common optimization techniques. This analysis enables a precise characterization of the GAMP algorithm fixed-points that applies to arbitrary transforms. In particular, we show that the fixed points of the so-called max-sum GAMP algorithm for MAP estimation are critical points of a constrained maximization of the posterior density. The fixed-points of the sum-product GAMP algorithm for estimation of the posterior marginals can be interpreted as critical points of a certain free energy.

show abstract

Section: Introductionmentioning

confidence: 99%

Fixed points of generalized approximate message passing with arbitrary matrices

Rangan¹,

Schniter

Riegler

et al. 2013

2013 IEEE International Symposium on Information Theory

Self Cite

View full text Add to dashboard Cite

show abstract

“…It should be noted that the same result can be derived in several different ways, including the approach on the basis of statistical mechanics of disordered systems [30], [31] and the study on the basis of a modified version of the null space characterization [32]- [34]. For further extensions, see e.g., [35]- [42].…”

Section: Random Matrix Ensemblesmentioning

confidence: 82%

A User's Guide to Compressed Sensing for Communications Systems

Hayashi

Nagahara

Tanaka

2013

IEICE Trans. Commun.

205

110

View full text Add to dashboard Cite

SUMMARYThis survey provides a brief introduction to compressed sensing as well as several major algorithms to solve it and its various applications to communications systems. We firstly review linear simultaneous equations as ill-posed inverse problems, since the idea of compressed sensing could be best understood in the context of the linear equations. Then, we consider the problem of compressed sensing as an underdetermined linear system with a prior information that the true solution is sparse, and explain the sparse signal recovery based on 1 optimization, which plays the central role in compressed sensing, with some intuitive explanations on the optimization problem. Moreover, we introduce some important properties of the sensing matrix in order to establish the guarantee of the exact recovery of sparse signals from the underdetermined system. After summarizing several major algorithms to obtain a sparse solution focusing on the 1 optimization and the greedy approaches, we introduce applications of compressed sensing to communications systems, such as wireless channel estimation, wireless sensor network, network tomography, cognitive radio, array signal processing, multiple access scheme, and networked control.

show abstract

“…The main advantage is that replica computations give a statistical physical meaning to the linear system. The replica method was used for compressed sensing in recent years [19][20][21][22]; we follow the research of [23].…”

Section: Convergence Discussionmentioning

confidence: 99%

Sparse Estimation Based on a New Random Regularized Matching Pursuit Generalized Approximate Message Passing Algorithm

Luo

Gui

Cong

et al. 2016

Entropy

View full text Add to dashboard Cite

Approximate Message Passing (AMP) and Generalized AMP (GAMP) algorithms usually suffer from serious convergence issues when the elements of the sensing matrix do not exactly match the zero-mean Gaussian assumption. To stabilize AMP/GAMP in these contexts, we have proposed a new sparse reconstruction algorithm, termed the Random regularized Matching pursuit GAMP (RrMpGAMP). It utilizes a random splitting support operation and some dropout/replacement support operations to make the matching pursuit steps regularized and uses a new GAMP-like algorithm to estimate the non-zero elements in a sparse vector. Moreover, our proposed algorithm can save much memory, be equipped with a comparable computational complexity as GAMP and support parallel computing in some steps. We have analyzed the convergence of this GAMP-like algorithm by the replica method and provided the convergence conditions of it. The analysis also gives an explanation about the broader variance range of the elements of the sensing matrix for this GAMP-like algorithm. Experiments using simulation data and real-world synthetic aperture radar tomography (TomoSAR) data show that our method provides the expected performance for scenarios where AMP/GAMP diverges.

show abstract

Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing

Cited by 173 publications

References 65 publications

Fixed points of generalized approximate message passing with arbitrary matrices

Fixed points of generalized approximate message passing with arbitrary matrices

A User's Guide to Compressed Sensing for Communications Systems

Sparse Estimation Based on a New Random Regularized Matching Pursuit Generalized Approximate Message Passing Algorithm

Contact Info

Product

Resources

About