Universality in polytope phase transitions and iterative algorithms

Abstract

We consider a class of nonlinear mappings FA,N in R N indexed by symmetric random matrices A ∈ R N ×N with independent entries. Within spin glass theory, special cases of these mappings correspond to iterating the TAP equations and were studied by Erwin Bolthausen. Within information theory, they are known as 'approximate message passing' algorithms. We study the high-dimensional (large N ) behavior of the iterates of F for polynomial functions F, and prove that it is universal, i.e. it depends only on the first two moments of the entries of A.As an application, we prove the universality of a certain phase transition arising in polytope geometry and compressed sensing. This solves a conjecture by David Donoho and Jared Tanner. I. MAIN RESULTSLet A ∈ R N ×N be a random Wigner matrix, i.e. a symmetric random matrix with i.i.d. entries (A ij ) i≤j satisfying E{A ij } = 0 and E{A 2 ij } = 1/N . A considerable effort has been devoted to studying the distribution of the eigenvalues of such a matrix [3]. The universality phenomenon is a striking recurring theme in these studies. Roughly speaking, many asymptotic properties of the joint eigenvalues distribution are independent of the entries distribution as long as it matches the first two moments, and satisfies certain tail conditions. We refer to [3] and references therein for a selection of such results. Universality is extremely useful because it allows to compute asymptotics for one entries distribution (e.g. Gaussian) and then export the results to a broad class of distributions.In this paper we are concerned with random matrix universality, albeit we do not focus onto eigenvalues properties. Given A ∈ R N ×N , and an initial condition x 0 ∈ R N independent of A, we consider the sequence (x t ) t∈N defined by letting, for t ≥ 0,Here, div denotes the divergence operator and, for each. The present paper is concerned with the asymptotic distribution of x t as N → ∞ with t fixed, and establishes the following results: Universality. As N → ∞, the finite-dimensional marginals of the distribution of x t are asymptotically insensitive to the distribution of the entries of A ij . State evolution. The entries of x t are asymptotically Gaussian with zero mean, and variance that can be explicitly computed through a recursion, known as state evolution Phase transitions in polytope geometry. As an application, we use state evolution to prove universality of a phase transition on polytope geometry, with connections to compressed sensing. This solves a conjecture put forward by David Donoho and Jared Tanner [6], [12].We think that the first two results provide a useful tool to establish a variety of universality in contexts in which the iteration (1) would not a priori seem relevant. The third of the above contributions demonstrates this. In fact, we shall consider a more general setting than the one just described, see Sections I-A and I-B.Iterations of the form (1) emerge in a number of contexts. In particular, approximate message passing algorithms (AMP) for compressed sensing ...

Decoding from pooled data: Phase transitions of message passing

“…We refer to [BM11,BLM12] for rigorous results, the assumptions of which do not apply to this setting. It is an interesting problem to prove the exactness of the SE equations in this setting.…”

Fixed points of generalized approximate message passing with arbitrary matrices

“…With some abuse of notation, we have used J SP (·) to denote both the LSL-BFE function in terms of q z as in (25) and the function in terms of the variance vector τ p as given in (30). Corresponding to (29), define the Lagrangian…”

Expectation-Maximization Gaussian-Mixture Approximate Message Passing

“…For this problem, accurate (relative to the noise variance) signal recovery is known to be possible with polynomial-complexity algorithms when is sufficiently sparse and when satisfies certain restricted isometry properties [4], or when is large with i.i.d. zero-mean sub-Gaussian entries [5] as discussed below.…”