The Convex algebraic geometry of linear inverse problems

Abstract

We study a class of ill-posed linear inverse problems in which the underlying model of interest has simple algebraic structure. We consider the setting in which we have access to a limited number of linear measurements of the underlying model, and we propose a general framework based on convex optimization in order to recover this model. This formulation generalizes previous methods based on 1 -norm minimization and nuclear norm minimization for recovering sparse vectors and low-rank matrices from a small number of linear measurements. For example some problems to which our framework is applicable include (1) recovering an orthogonal matrix from limited linear measurements, (2) recovering a measure given random linear combinations of its moments, and (3) recovering a low-rank tensor from limited linear observations. I. INTRODUCTIONA fundamental principle in the study of ill-posed inverse problems is the incorporation of prior information. Such prior information typically specifies some sort of simplicity present in the model. This simplicity could concern the number of genes present in a signature for disease, correlation structures in time series data, or geometric constraints on molecular configurations. Unfortunately, naive mathematical characterizations of model simplicity often result in intractable optimization problems. This paper formalizes a methodology for transforming notions of simplicity into convex penalty functions. These penalty functions are induced by the convex hulls of simple models. We show that algorithms resulting from these formulations allow for information extraction with the number of measurements scaling as a function of the number of latent degrees of freedom of the simple model. We provide a methodology for estimating these scaling laws and discuss several examples. We focus on situations in which the basic models are semialgebraic, in which case the regularized inverse problems can be solved or approximated by semidefinite programming.Specifically suppose that there is an object of interest x ∈ R p that is formed as the linear combination of a small number of elements of a basic semi-algebraic set A :

Equivalence of synthesis and atomic formulations of sparse recovery

“…In this paper, we reveal the connection between the works in [3] and [6] by proving that the synthesis and atomic norm formulations are equivalent. This equivalence has many important implications: First, it allows an alternative geometric way (e.g., see [7]) of obtaining tight measurement bounds for the exact recovery that can be used in synthesis formulation and vice versa.…”

“…This equivalence has many important implications: First, it allows an alternative geometric way (e.g., see [7]) of obtaining tight measurement bounds for the exact recovery that can be used in synthesis formulation and vice versa. Second, it finds the solution to the open question in [6] by suggesting the synthesis formulation for recovery of the sparse coefficients of the signal with the least atomic norm. Moreover, it enables one to solve the minimization problem in a lower dimensional space R n in the cases that the atoms in the dictionary have a simple algebraic structure, instead of the 1 minimization, which may have to operate on a higher (possibly infinite) dimensional space R l .…”

“…In the CS literature, Rauhut et al [4] and Candes et al [5] established algorithmic guarantees for the synthesis and analysis formulations. Now, there is a new formulation in town, proposed by Chandrasekaran et al in [6], which we refer to as the atomic formulation of sparse recoveryá la Elad et al The authors in [6] introduce an algorithmic way for recovering the signal x (or its approximations) by doing minimization over a norm induced by the geometry of the dictionary. This signal recovery formulation-seemingly-differ from the synthesis formulation, and cannot recover the sparse coefficients in general.…”

“…This signal recovery formulation-seemingly-differ from the synthesis formulation, and cannot recover the sparse coefficients in general. A key strength of this approach is the associated geometric analysis techniques which enables one to find tight upper bounds on the required number of measurements for perfect recovery using random measurements based on Gaussian width calculations [6].…”

Multi-structural Signal Recovery for Biomedical Compressive Sensing

“…Sparsity is assumed in one domain as the key constraint to recover the signal [5]. Recently, progress shows that other structure information can be exploited to recover signals [6] [7], such as piecewise smoothness [8], low-rank property [9], [10], orthogonality [7], permutation [7].…”