High-Dimensional Estimation of Structured Signals From Non-Linear Observations With General Convex Loss Functions

Genzel, Martin

doi:10.1109/tit.2016.2642993

Cited by 38 publications

(46 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The squared loss then just corresponds to L(v 1 , v 2 ) = 1 2 (v 1 − v 2 ) 2 . Under relatively mild conditions on L, such as restricted strong convexity, similar recovery guarantees as above can be proven; see again [Gen17].…”

Section: Further Extensionsmentioning

confidence: 66%

“…A somewhat astonishing observation of [PV16;Gen17] was that recovery from single-index observations (1.2) is already possible by means of the vanilla Lasso [Tib96], even though the non-linear distortion f is completely unknown. As we shall see next, such a strategy can be also adapted to the more advanced measurement scheme of (1.4).…”

Section: Algorithmic Approachesmentioning

confidence: 99%

See 1 more Smart Citation

Recovering Structured Data From Superimposed Non-Linear Measurements

Genzel

Jung

2020

IEEE Trans. Inform. Theory

Self Cite

View full text Add to dashboard Cite

This work deals with the problem of distributed data acquisition under non-linear communication constraints. More specifically, we consider a model setup where M distributed nodes take individual measurements of an unknown structured source vector x 0 ∈ R n , communicating their readings simultaneously to a central receiver. Since this procedure involves collisions and is usually imperfect, the receiver measures a superposition of non-linearly distorted signals. In a first step, we will show that an s-sparse vector x 0 can be successfully recovered from O(s · log(2n/s)) of such superimposed measurements, using a traditional Lasso estimator that does not rely on any knowledge about the non-linear corruptions. This direct method however fails to work for several "uncalibrated" system configurations. These blind reconstruction tasks can be easily handled with the 1,2 -Group Lasso, but coming along with an increased sampling rate of O(s · max{M, log(2n/s)}) observations-in fact, the purpose of this lifting strategy is to extend a certain class of bilinear inverse problems to non-linear acquisition. Apart from that, we shall also demonstrate how our two algorithmic approaches can be embedded into a more flexible and adaptive recovery framework which allows for sub-Gaussian measurement designs as well as general (convex) structural constraints. Finally, to illustrate the practical scope of our theoretical findings, an application to wireless sensor networks is discussed, which actually serves as the prototype example of our observation model.

show abstract

Section: Further Extensionsmentioning

confidence: 66%

Section: Algorithmic Approachesmentioning

confidence: 99%

Recovering Structured Data From Superimposed Non-Linear Measurements

Genzel

Jung

2020

IEEE Trans. Inform. Theory

Self Cite

View full text Add to dashboard Cite

show abstract

“…One can also imagine extending our results to general quantization schemes, e.g, general number of quantization levels [TAH15]. Finally, considering other loss functions in (3) (e.g., see [Gen17]) and the performance of first-order solvers (e.g., see [OS16]) are also interesting directions to pursue. Mark…”

Section: Future Workmentioning

confidence: 99%

The Generalized Lasso for Sub-Gaussian Measurements With Dithered Quantization

Thrampoulidis

Rawat

2020

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

In the problem of structured signal recovery from high-dimensional linear observations, it is commonly assumed that full-precision measurements are available. Under this assumption, the recovery performance of the popular Generalized Lasso (G-Lasso) is by now well-established. In this paper, we extend these types of results to the practically relevant settings with quantized measurements. We study two extremes of the quantization schemes, namely, uniform and one-bit quantization; the former imposes no limit on the number of quantization bits, while the second only allows for one bit. In the presence of a uniform dithering signal and when measurement vectors are sub-gaussian, we show that the same algorithm (i.e., the G-Lasso) has favorable recovery guarantees for both uniform and one-bit quantization schemes. Our theoretical results, shed light on the appropriate choice of the range of values of the dithering signal and accurately capture the error dependence on the problem parameters. For example, our error analysis shows that the G-Lasso with one-bit uniformly dithered measurements leads to only a logarithmic rate loss compared to the fullprecision measurements.

show abstract

“…Despite the universal applicability of the Lasso, practitioners however often choose different types of loss functions for (P L,K ), which are specifically tailored to their model hypotheses, e.g., if the output variables y i are discrete. This issue particularly motivated the first author in [Gen17] to extent the framework of Plan and Vershynin to other choices of L. A key finding of [Gen17] is that, in many situations of interest, restricted strong convexity (RSC) is a crucial property of an empirical risk function to ensure successful signal recovery via (P L,K ). The criterion of RSC is indeed satisfied for a large class of loss functions, for instance, all those L : R × R → R which are twice differentiable in the first variable and locally strongly convex in a neighborhood of the origin (cf.…”

Section: Signal Processing and Compressed Sensingmentioning

confidence: 99%

Robust 1-bit compressed sensing via hinge loss minimization

Genzel

Stollenwerk

2019

Information and Inference: A Journal of the IMA

Self Cite

View full text Add to dashboard Cite

This work theoretically studies the problem of estimating a structured high-dimensional signal x 0 ∈ R n from noisy 1-bit Gaussian measurements. Our recovery approach is based on a simple convex program which uses the hinge loss function as data fidelity term. While such a risk minimization strategy is very natural to learn binary output models, such as in classification, its capacity to estimate a specific signal vector is largely unexplored. A major difficulty is that the hinge loss is just piecewise linear, so that its "curvature energy" is concentrated in a single point. This is substantially different from other popular loss functions considered in signal estimation, e.g., the square or logistic loss, which are at least locally strongly convex. It is therefore somewhat unexpected that we can still prove very similar types of recovery guarantees for the hinge loss estimator, even in the presence of strong noise. More specifically, our non-asymptotic error bounds show that stable and robust reconstruction of x 0 can be achieved with the optimal oversampling rate O(m −1/2 ) in terms of the number of measurements m. Moreover, we permit a wide class of structural assumptions on the ground truth signal, in the sense that x 0 can belong to an arbitrary bounded convex set K ⊂ R n . The proofs of our main results rely on some recent advances in statistical learning theory due to Mendelson. In particular, we invoke an adapted version of Mendelson's small ball method that allows us to establish a quadratic lower bound on the error of the first order Taylor approximation of the empirical hinge loss function. 1 Hereafter, we make the convention that sign(v) = +1 for v ≥ 0 and sign(v) = −1 for v < 0. arXiv:1804.04846v2 [math.ST]

show abstract

High-Dimensional Estimation of Structured Signals From Non-Linear Observations With General Convex Loss Functions

Cited by 38 publications

References 50 publications

Recovering Structured Data From Superimposed Non-Linear Measurements

Recovering Structured Data From Superimposed Non-Linear Measurements

The Generalized Lasso for Sub-Gaussian Measurements With Dithered Quantization

Robust 1-bit compressed sensing via hinge loss minimization

Contact Info

Product

Resources

About