Message-passing algorithms for compressed sensing

Abstract

Compressed sensing aims to undersample certain high-dimensional signals yet accurately reconstruct them by exploiting signal characteristics. Accurate reconstruction is possible when the object to be recovered is sufficiently sparse in a known basis. Currently, the best known sparsity-undersampling tradeoff is achieved when reconstructing by convex optimization, which is expensive in important large-scale applications. Fast iterative thresholding algorithms have been intensively studied as alternatives to convex optimization for large-scale problems. Unfortunately known fast algorithms offer substantially worse sparsity-undersampling tradeoffs than convex optimization. We introduce a simple costless modification to iterative thresholding making the sparsityundersampling tradeoff of the new algorithms equivalent to that of the corresponding convex optimization procedures. The new iterative-thresholding algorithms are inspired by belief propagation in graphical models. Our empirical measurements of the sparsity-undersampling tradeoff for the new algorithms agree with theoretical calculations. We show that a state evolution formalism correctly derives the true sparsity-undersampling tradeoff. There is a surprising agreement between earlier calculations based on random convex polytopes and this apparently very different theoretical formalism. combinatorial geometry | phase transitions | linear programming | iterative thresholding algorithms | state evolution C ompressed sensing refers to a growing body of techniques that "undersample" high-dimensional signals and yet recover them accurately (1). Such techniques make fewer measurements than traditional sampling theory demands: rather than sampling proportional to frequency bandwidth, they make only as many measurements as the underlying "information content" of those signals. However, compared with traditional sampling theory, which can recover signals by applying simple linear reconstruction formulas, the task of signal recovery from reduced measurements requires nonlinear and, so far, relatively expensive reconstruction schemes. One popular class of reconstruction schemes uses linear programming (LP) methods; there is an elegant theory for such schemes promising large improvements over ordinary sampling rules in recovering sparse signals. However, solving the required LPs is substantially more expensive in applications than the linear reconstruction schemes that are now standard. In certain imaging problems, the signal to be acquired may be an image with 10 6 pixels and the required LP would involve tens of thousands of constraints and millions of variables. Despite advances in the speed of LP, such problems are still dramatically more expensive to solve than we would like.Here, we develop an iterative algorithm achieving reconstruction performance in one important sense identical to LP-based reconstruction while running dramatically faster. We assume that a vector y of n measurements is obtained from an unknown Nvector x 0 according to y = Ax 0 , where A is the n ...

Statistical estimation: from denoising to sparse regression and hidden cliques

“…Since G(0; 0) = 0, we know that τ = 0 is always a fixed point. It is not hard to shown [16] that indeed lim t→∞ τ 2 t = 0 if and only if this is the unique non-negative fixed point, see figure below. If this condition is satisfied, AMP reconstructs exactly the signal θ, and due to the correspondence with the LASSO, also basis pursuit (the LASSO with λ → 0) reconstructs exactly θ.…”

“…Explicit formulae for M (ε) can be found -for instance-in [16,Supplementary Material] or [17]. A sketch is shown in Fig.…”

“…We will sketch the main ideas referring to [16] for the original idea, to [17,41,42] for the analysis of the LASSO, and to [7,43,44] for extensions. This approach was also used in [45] to establish universality of the compressed sensing phase transition for non-Gaussian i.i.d.…”

Ultrasound image reconstruction from compressed measurements using approximate message passing

“…Inspired by belief propagation techniques, the approximate message passing (AMP) algorithm has been introduced in [4] as an alternative to CS reconstruction techniques that are based on minimizing (2) or similar objective functions. At each iteration, AMP consists of two steps as shown in equations (3) and (4).…”

“…The objective of this paper is to propose enhanced CS recovery algorithms for compressively sampled US images, compared to previously proposed algorithms. The proposed method is based on the Approximate Message Passing (AMP) algorithm, a CS recovery technique that turns the reconstruction problem into an iterative denoising approach [4][5][6]. This paper focusses on the selection of a relevant sparsifying basis and a robust denoiser in order to maximize the performance of AMP in ultrasound CS reconstruction.…”