Statistical-Physics-Based Reconstruction in Compressed Sensing

Abstract-Signal recovery from unitarily invariant measurements is investigated in this paper. A message-passing algorithm is formulated on the basis of expectation propagation (EP). A rigorous analysis is presented for the dynamics of the algorithm in the large system limit, where both input and output dimensions tend to infinity while the compression rate is kept constant. The main result is the justification of state evolution (SE) equations conjectured by Ma and Ping. This result implies that the EPbased algorithm achieves the Bayes-optimal performance derived by Takeda et al. in 2006 via a non-rigorous tool in statistical physics, when the compression rate is larger than a threshold. The proof is based on an extension of a conventional conditioning technique for the standard Gaussian matrix to the case of the Haar matrix.

show abstract

“…Furthermore, let mse t n denote the individual MSE for the estimate (14) in the EP-based algorithm in the large system limit, given by…”

Section: Resultsmentioning

confidence: 99%

“…(57) (d) The individual MSEs (33) and (32) for the posterior and extrinsic estimates (14) and (17) in module B coincide with the MMSE (15) and extrinsic variance (18) in the large system limit,…”

Section: = Limmentioning

confidence: 99%

Rigorous dynamics of expectation-propagation-based signal recovery from unitarily invariant measurements

Takeuchi

2017

2017 IEEE International Symposium on Information Theory (ISIT)

104

202

View full text Add to dashboard Cite

show abstract

“…GAMP-based methods have also been extended in a wide variety of ways, such as combining EM with GAMP [58]- [62], turbo and hybrid GAMP methods [63], [64], applications in dictionary learning and matrix factorization [65]- [69], and applications in blind deconvolution and self-calibration [70]. Another line of work would be to understand if one can find free energy and optimization interpretations of these algorithms.…”

Section: Discussionmentioning

confidence: 99%

Fixed points of generalized approximate message passing with arbitrary matrices

Rangan¹,

Schniter

Riegler

et al. 2013

2013 IEEE International Symposium on Information Theory

View full text Add to dashboard Cite

Abstract-The estimation of a random vector with independent components passed through a linear transform followed by a componentwise (possibly nonlinear) output map arises in a range of applications. Approximate message passing (AMP) methods, based on Gaussian approximations of loopy belief propagation, have recently attracted considerable attention for such problems. For large random transforms, these methods exhibit fast convergence and admit precise analytic characterizations with testable conditions for optimality, even for certain non-convex problem instances. However, the behavior of AMP under general transforms is not fully understood. In this paper, we consider the Generalized AMP (GAMP) algorithm and relate the method to more common optimization techniques. This analysis enables a precise characterization of the GAMP algorithm fixed-points that applies to arbitrary transforms. In particular, we show that the fixed points of the so-called max-sum GAMP algorithm for MAP estimation are critical points of a constrained maximization of the posterior density. The fixed-points of the sum-product GAMP algorithm for estimation of the posterior marginals can be interpreted as critical points of a certain free energy.

show abstract

“…The algorithm relies on the Central Limit Theorem to simplify loopy belief propagation by replacing continuous-domain convolutions with matrix-vector multiplications followed by pointwise nonlinearities [6,13]. Although the basic GAMP algorithm requires perfect knowledge of S, λ, and v w for reconstruction, it was recently extended to incorporate parameter learning [7][8][9]. In particular, a recently introduced adaptive GAMP method combines maximum-likelihood (ML) estimation with the standard GAMP updates [7].…”

Section: Calibration With Adaptive Gampmentioning

confidence: 99%

“…Recently, a generalized version of GAMP called adaptive GAMP was proposed for solving inverse problems where the signal and noise distributions have parametric uncertainties [7]. Adaptive GAMP generalizes several prior works [8,9], and provably yields asymptotically consistent estimates of the unknown parameters. In the following, we demonstrate that the adaptive GAMP framework can be extended in order to solve (1).…”

Section: Introductionmentioning

confidence: 99%

Autocalibrated signal reconstruction from linear measurements using adaptive GAMP

Kamilov

Bourquard

Bostan

et al. 2013

2013 IEEE International Conference on Acoustics, Speech and Signal Processing

View full text Add to dashboard Cite

In this paper, we reconstruct signals from underdetermined linear measurements where the componentwise gains of the measurement system are unknown a priori. The reconstruction is performed through an adaptation of the messagepassing algorithm called adaptive GAMP that enables joint gain calibration and signal estimation. To evaluate our approach, we apply it to the problem of sparse recovery and compare it against an 1 -based approach. We numerically show that adaptive GAMP yields excellent results even for a moderate amount of data. It approaches the performance of oracle GAMP where the gains are perfectly known asymptotically.

show abstract

Statistical-Physics-Based Reconstruction in Compressed Sensing

Cited by 251 publications

References 40 publications

Rigorous dynamics of expectation-propagation-based signal recovery from unitarily invariant measurements

Rigorous dynamics of expectation-propagation-based signal recovery from unitarily invariant measurements

Fixed points of generalized approximate message passing with arbitrary matrices

Autocalibrated signal reconstruction from linear measurements using adaptive GAMP

Contact Info

Product

Resources

About