A Simple Derivation of AMP and its State Evolution via First-Order Cancellation

Abstract

We consider the linear regression problem, where the goal is to recover the vector x ∈ R n from measurements y = Ax + w ∈ R m under known matrix A and unknown noise w. For large i.i.d. sub-Gaussian A, the approximate message passing (AMP) algorithm is precisely analyzable through a state-evolution (SE) formalism, which furthermore shows that AMP is Bayes optimal in certain regimes. The rigorous SE proof, however, is long and complicated. And, although the AMP algorithm can be derived as an approximation of loop belief propagation (LBP), this viewpoint provides little insight into why large i.i.d. A matrices are important for AMP, and why AMP has a state evolution. In this work, we provide a heuristic derivation of AMP and its state evolution, based on the idea of "firstorder cancellation," that provides insights missing from the LBP derivation while being much shorter than the rigorous SE proof. * P. Schniter (email: schniter.1@osu.edu) is with the

A Hybrid Approach to Coded Compressed Sensing Where Coupling Takes Place Via the Outer Code

“…Specifically, outer codewords are partitioned into L = 16 blocks, each of length 16 bits. After switching to the index representation, section m( ) has length 2 16 , yielding an overall vector m of length L2 16 = 2 20 . Both the block diagonal matrix for CCS and CCS-hybrid, and the dense matrix for the two CCS-AMP variants are formed by selecting random rows from Hadamard matrices (excluding the row of all ones).…”

“…We note that τ 2 t can be approximated as τ 2 t ≈ z (t) 2 /n for t ≥ 0. A justification for the same can be found in [20]. Message passing over large sections can be performed efficiently using the Fast Fourier transform or the Fast Walsh-Hadamard transform for a suitably designed outer code as in [10].…”

“…In this sense, it is an AMP-inspired framework that seeks to reap the benefits of a dense matrix at a much lower complexity. It is worth noting that, although CCS-AMP is designed to handle very large dimensions, it is constrained in that the width of A should not exceed 2 20 or so. In the hybrid implementation the state update is applied block-wise after denoising and, hence, the width of a block is constrained but not the width of the overall block diagonal sensing matrix.…”

User Activity Detection and Channel Estimation of Spatially Correlated Channels via AMP in Massive MTC

“…This paper considers that the pilot sequences are generated from a complex symmetric Bernoulli distribution. This setup is motivated by the fact that: 1) pilot sequences generated from a complex symmetric Bernoulli distribution are practical as they can be deployed using quadratic phase shift keying (QPSK) modulation, 2) matrices drawn from a Bernoulli distribution are well suited for AMP-based support and signal recovery [5], [15] as we will discuss later.…”

A unifying tutorial on Approximate Message Passing

“…The statistical roots of AMP lie in compressed sensing (Donoho et al, 2009(Donoho et al, , 2013. A reader approaching the subject from this perspective can consult Montanari (2012), Tramel et al (2014) and Schniter (2020) for accessible expositions of the motivating ideas and the connections with message passing algorithms on dense graphs. Alternatively, for comprehensive reviews of AMP from a physics perspective, see Zdeborová and Krzakala (2016), Krzakala et al (2012) and Lesieur et al (2017).…”