Variational free energies for compressed sensing

Krząkała, Florent; Manoel, Andre; Tramel, Eric W.; Zdeborová, Lenka

doi:10.1109/isit.2014.6875083

Cited by 47 publications

(69 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Under the posterior approximation b X and large i.i.d A, [53] claims that the negative log likelihood − ln p(y; ν w ) is well approximated by a Bethe free entropy of the form …”

Section: Appendix C Em Update For Noise Variancementioning

confidence: 99%

Compressive phase retrieval via generalized approximate message passing

Schniter

Rangan

2012

2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton)

145

199

View full text Add to dashboard Cite

Abstract-In phase retrieval, the goal is to recover a signal x ∈ C N from the magnitudes of linear measurements Ax ∈ C M . While recent theory has established that M ≈ 4N intensity measurements are necessary and sufficient to recover generic x, there is great interest in reducing the number of measurements through the exploitation of sparse x, which is known as compressive phase retrieval. In this work, we detail a novel, probabilistic approach to compressive phase retrieval based on the generalized approximate message passing (GAMP) algorithm. We then present a numerical study of the proposed PR-GAMP algorithm, demonstrating its excellent phase-transition behavior, robustness to noise, and runtime. Our experiments suggest that approximately M ≥ 2K log 2 (N/K) intensity measurements suffice to recover K-sparse BernoulliGaussian signals for A with i.i.d Gaussian entries and K ≪ N . Meanwhile, when recovering a 6k-sparse 65k-pixel grayscale image from 32k randomly masked and blurred Fourier intensity measurements at 30 dB measurement SNR, PR-GAMP achieved an output SNR of no less than 28 dB in all of 100 random trials, with a median runtime of only 7.3 seconds. Compared to the recently proposed CPRL, sparse-Fienup, and GESPAR algorithms, our experiments suggest that PR-GAMP has a superior phase transition and orders-of-magnitude faster runtimes as the sparsity and problem dimensions increase.

show abstract

“…Under the posterior approximation b X and large i.i.d A, [53] claims that the negative log likelihood − ln p(y; ν w ) is well approximated by a Bethe free entropy of the form …”

Section: Appendix C Em Update For Noise Variancementioning

confidence: 99%

Compressive phase retrieval via generalized approximate message passing

Schniter

Rangan

2012

2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton)

145

199

View full text Add to dashboard Cite

show abstract

“…Although AMP is a powerful method, it does not always converge to a solution. According to [31], the convergence properties of the AMP algorithm can be increased by estimating the variances of the noise and sparsity error. In our proposed algorithm, ∆ and Υ can be estimated by using the Bethe free energy.…”

Section: Noise Variance Learningmentioning

confidence: 99%

Beamspace Channel Estimation in mmWave Systems via Cosparse Image Reconstruction Technique

Yang

Wen

Jin

et al. 2018

IEEE Trans. Commun.

View full text Add to dashboard Cite

This paper considers the beamspace channel estimation problem in 3D lens antenna array under a millimeter-wave communication system. We analyze the focusing capability of the 3D lens antenna array and the sparsity of the beamspace channel response matrix. Considering the analysis, we observe that the channel matrix can be treated as a 2D natural image; that is, the channel is sparse, and the changes between adjacent elements are subtle. Thus, for the channel estimation, we incorporate an image reconstruction technique called sparse non-informative parameter estimator-based cosparse analysis Ā M P for īmaging (SCAMPI) algorithm. The SCAMPI algorithm is faster and more accurate than earlier algorithms such as orthogonal matching pursuit and support detection algorithms. To further improve the SCAMPI algorithm, we model the channel distribution as a generic Gaussian mixture (GM) probability and embed the expectation maximization learning algorithm into the SCAMPI algorithm to learn the parameters in the GM probability. We show that the GM probability outperforms the common uniform distribution used in image reconstruction. We also propose a phase-shifter-reduced selection network structure to decrease the power consumption of the system and prove that the SCAMPI algorithm is robust even if the number of phase shifters is reduced by 10%. Index TermsMillimeter wave communication system, lens antenna array, SCAMPI algorithm, GM probability, EM learning.

show abstract

“…Unfortunately, for the GLM model in the previous section, the computations in loopy BP may be difficult for dense matrices A. The sum-product GAMP algorithm from [4] can be interpreted as a method for minimizing an approximation of the BFE that applies to certain large, dense A [14], [15]. Specifically, the sum-product GAMP algorithm finds estimates b x (x) and b z (z) of the posterior densities p(x|y) and p(z|y) via the minimization,…”

Section: A Bethe Free Energy Minimizationmentioning

confidence: 99%

“…The objective function of the optimization in (6) can be interpreted as an approximation of the BFE for the GLM from Section I in a certain large-system limit, where m, n → ∞ and A has i.i.d. components [15]. We thus call this approximate BFE the large-system limit Bethe Free Energy or LSL-BFE.…”

Section: A Bethe Free Energy Minimizationmentioning

confidence: 99%

“…Our approach to stabilizing GAMP is based on reconsidering the inference problem as a type of free-energy minimization. Specifically, it is known that GAMP can be considered as an iterative procedure that attempts to minimize a large-system-limit approximation of the so-called Bethe Free Energy (BFE) [14], [15], which we abbreviate as "LSL-BFE" in the sequel.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Inference for Generalized Linear Models via Alternating Directions and Bethe Free Energy Minimization

Rangan

Fletcher

Schniter

et al. 2017

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

Abstract-Generalized Linear Models (GLMs), where a random vector x is observed through a noisy, possibly nonlinear, function of a linear transform z = Ax arise in a range of applications in nonlinear filtering and regression. Approximate Message Passing (AMP) methods, based on loopy belief propagation, are a promising class of approaches for approximate inference in these models. AMP methods are computationally simple, general, and admit precise analyses with testable conditions for optimality for large i.i.d. transforms A. However, the algorithms can easily diverge for general transforms. This paper presents a convergent approach to the generalized AMP (GAMP) algorithm based on direct minimization of a large-system limit approximation of the Bethe Free Energy (LSL-BFE). The proposed method uses a double-loop procedure, where the outer loop successively linearizes the LSL-BFE and the inner loop minimizes the linearized LSL-BFE using the Alternating Direction Method of Multipliers (ADMM). The proposed method, called ADMM-GAMP, is similar in structure to the original GAMP method, but with an additional least-squares minimization. It is shown that for strictly convex, smooth penalties, ADMM-GAMP is guaranteed to converge to a local minima of the LSL-BFE, thus providing a convergent alternative to GAMP that is stable under arbitrary transforms. Simulations are also presented that demonstrate the robustness of the method for non-convex penalties as well.

show abstract

Variational free energies for compressed sensing

Cited by 47 publications

References 19 publications

Compressive phase retrieval via generalized approximate message passing

Compressive phase retrieval via generalized approximate message passing

Beamspace Channel Estimation in mmWave Systems via Cosparse Image Reconstruction Technique

Inference for Generalized Linear Models via Alternating Directions and Bethe Free Energy Minimization

Contact Info

Product

Resources

About