A Unified Approach to Uniform Signal Recovery From Nonlinear Observations

Quantizing Heavy-Tailed Data in Statistical Estimation: (Near) Minimax Rates, Covariate Quantization, and Uniform Recovery

“…1) Like the recent work [40], one crucial technical tool in our proof is a powerful concentration inequality for product process due to Mendelson [67], as adapted in the present Lemma 9. However, [40] only studied sub-Gaussian distribution, and the results produced by their unified approach typically exhibit a decaying rate of O(n −1/4 ) [40,Sect. 4].…”

“…More precisely, a uniform guarantee ensures the recovery of all structured signals of interest with a single draw of the sensing ensemble, while a non-uniform guarantee is only valid for a structured signal fixed before drawing the random ensemble, with the implication that a new realization of the sensing matrix is required for sensing a new signal. Uniformity is a highly desired property in compressed sensing, since in applications the measurement ensemble is typically fixed and is expected to work for all signals [21], [40]. Besides, the derivation of a uniform guarantee is often significantly harder than a non-uniform one, making uniformity an interesting theoretical problem in its own right.…”

“…1.3]). It is worth noting that the interesting recent work [40] provided a unified approach to uniform guarantee for a series of non-linear models; however, their uniform guarantees typically exhibit a decaying rate of O(n −1/4 ) that is slower than the non-uniform one of O(n −1/2 ) (Section 4 therein). As a follow-up work, [21] obtained a near optimal uniform O(n −1/2 ) error rate for various nonlinear generative compressed sensing models.…”

“…This is done by upgrading Theorem 5 to be uniform over all sparse θ ⋆ by more in-depth technical tools and a careful covering argument. Part of the techniques is inspired by prior works [40], [94], but certain technical innovations are required:…”

“…We follow most assumptions in Theorem 5 but specify the signal space as θ ⋆ ∈ Σ s,R0 and impose the (2l)th moment constraint on ϵ k . Following prior works on QCS (e.g., [40], [89]), we consider constrained Lasso that utilizes an ℓ 1 -constraint ∥θ∥ 1 ≤ ∥θ ⋆ ∥ 1 (rather than (10)) to pursue uniform recovery.…”