The squared-error of generalized LASSO: A precise analysis

Abstract

We consider the problem of estimating an unknown but structured signal x0 from its noisy linear observations y = Ax0 + z ∈ R m . To the structure of x0 is associated a structure inducing convex function f (·). We assume that the entries of A are i.i.d. standard normal N (0, 1) and z ∼ N (0, σ 2 Im). As a measure of performance of an estimate x * of x0 we consider the "Normalized Square Error" (NSE) x * − x0 2 2 /σ 2 . For sufficiently small σ, we characterize the exact performance of two different versions of the well known LASSO algorithm. The first estimator is obtained by solving the problem arg minx y − Ax 2 + λf (x). As a function of λ, we identify three distinct regions of operation. Out of them, we argue that "RON " is the most interesting one. When λ ∈ RON , we show that the NSE isis the expected squared-distance of an i.i.d. standard normal vector to the dilated subdifferential λ · ∂f (x0). Secondly, we consider the more popular estimator arg minx 1 2 y − Ax 2 2 + στ f (x). We propose a formula for the NSE of this estimator by establishing a suitable mapping between this and the previous estimator over the region RON . As a useful side result, we find explicit formulae for the optimal estimation performance and the optimal penalty parameters λ * and τ * .

A case for orthogonal measurements in linear inverse problems

“…On the other hand, when there is noise, as the restricted eigenvalue min a∈D 1 (x0) Aa 2 gets larger it is known that the recovery is robuster and one suffers from less error, [4], [12], [14]. Applying Theorem 2 on the set D 1 (x 0 ) ensures that, restricted eigenvalues of URS's are as good as i.i.d.…”

The Generalized Lasso With Non-Linear Observations

“…This showed that previous results were tight. A line of work by Thrampoulidis, Oymak, and Hassibi [33,34,42] concentrated on the precise reconstruction error from noisy observations, and also considered unconstrained versions of the K-Lasso. Our theoretical results in the non-linear case can be seen to mirror Theorem [33, Theorem 1] in the linear case.…”

A Simple Tool for Bounding the Deviation of Random Matrices on Geometric Sets

“…Predating, but especially following, the works in compressed sensing, there have also been several works which tackle the general case, giving results for arbitrary T [11,25,17,16,1,3,20,21]. The deviation inequalities of this paper allow for a general treatment as well.…”