A totally unimodular view of structured sparsity

Halabi, Marwa El; Cevher, Volkan

doi:10.48550/arxiv.1411.1990

Cited by 2 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…the ℓ 1 -norm for sparse signals, the nuclear-norm for low-rank ones, etc. Please refer for example to [CRPW12], [Bac10], [HC14], [ALMT13] for further discussions.…”

Section: Introductionmentioning

confidence: 99%

The LASSO with Non-linear Measurements is Equivalent to One With Linear Measurements

Thrampoulidis,

Abbasi,

Hassibi

2015

Preprint

View full text Add to dashboard Cite

Consider estimating an unknown, but structured (e.g. sparse, low-rank, etc.), signal x 0 ∈ R n from a vector y ∈ R m of measurements of the form y i = g i (a i T x 0 ), where the a i 's are the rows of a known measurement matrix A, and, g(•) is a (potentially unknown) nonlinear and random link-function. Such measurement functions could arise in applications where the measurement device has nonlinearities and uncertainties. It could also arise by design, e.g., g i (x) = sign(x + z i ), corresponds to noisy 1-bit quantized measurements. Motivated by the classical work of Brillinger, and more recent work of Plan and Vershynin, we estimate x 0 via solving the Generalized-LASSO, i.e., x := arg min x y − Ax 0 2 + λf (x) for some regularization parameter λ > 0 and some (typically non-smooth) convex regularizer f (•) that promotes the structure of x 0 , e.g. ℓ 1 -norm, nuclear-norm, etc. While this approach seems to naively ignore the nonlinear function g(•), both Brillinger (in the non-constrained case) and Plan and Vershynin have shown that, when the entries of A are iid standard normal, this is a good estimator of x 0 up to a constant of proportionality µ, which only depends on g(•). In this work, we considerably strengthen these results by obtaining explicit expressions for x − µx 0 2 , for the regularized Generalized-LASSO, that are asymptotically precise when m and n grow large. A main result is that the estimation performance of the Generalized LASSO with non-linear measurements is asymptotically the same as one whose measurements are linear y i = µa i T x 0 + σz i , with µ = Eγg(γ) and σ 2 = E(g(γ) − µγ) 2 , and, γ standard normal. To the best of our knowledge, the derived expressions on the estimation performance are the first-known precise results in this context. One interesting consequence of our result is that the optimal quantizer of the measurements that minimizes the estimation error of the Generalized LASSO is the celebrated Lloyd-Max quantizer.

show abstract

“…the ℓ 1 -norm for sparse signals, the nuclear-norm for low-rank ones, etc. Please refer for example to [CRPW12], [Bac10], [HC14], [ALMT13] for further discussions.…”

Section: Introductionmentioning

confidence: 99%

The LASSO with Non-linear Measurements is Equivalent to One With Linear Measurements

Thrampoulidis,

Abbasi,

Hassibi

2015

Preprint

View full text Add to dashboard Cite

show abstract

“…In this paper, we investigate the statistical properties of the Exclusive Lasso for sparse, within group variable selection in regression problems. Specifically, our novel contributions beyond the existing literature (Zhou et al, 2010;Obozinski and Bach, 2012;Halabi and Cevher, 2014) include: characterizing the Exclusive Lasso solution and relating this solution to the existing statistics literature on penalized regression (Section 2); proving consistency and prediction consistency (Section 3); developing a fast algorithm with convergence guarantees for estimation (Section 4); deriving the degrees of freedom that can be used for model selection (Section 5); and investigating the empirical performance of our method through simulations (Sections 6 and 7).…”

Section: Introductionmentioning

confidence: 95%

“…Even though this problem is known to be NP-hard, a popular approach in the literature uses convex penalties to relax similar combinatorial problems into tractable convex problems. While the Lasso (Tibshirani, 1996) is the most well known of these convex relaxations, there are several frameworks specifically designed to find convex alternatives to complicated structured combinatorial problems (Obozinski and Bach, 2012;Halabi and Cevher, 2014). These frameworks lead to convex penalties like the Group Lasso (Yuan and Lin, 2006), Composite Absolute Penalties (Zhao et al, 2009), and the Exclusive Lasso (Zhou et al, 2010), the subject of this paper.…”

Section: Introductionmentioning

confidence: 99%

“…These frameworks lead to convex penalties like the Group Lasso (Yuan and Lin, 2006), Composite Absolute Penalties (Zhao et al, 2009), and the Exclusive Lasso (Zhou et al, 2010), the subject of this paper. Zhou et al (2010) first uses the Exclusive Lasso penalty in the context of multitask learning, and Obozinski and Bach (2012) and Halabi and Cevher (2014) relate the penalty to their framework for relaxing combinatorial problems. The Exclusive Lasso penalty has not yet been explored statistically or developed into a method that can be used for sparse regression and within group variable selection.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Within group variable selection through the Exclusive Lasso

Campbell¹,

Allen²

2017

Electron. J. Statist.

View full text Add to dashboard Cite

Many data sets consist of variables with an inherent group structure. The problem of group selection has been well studied, but in this paper, we seek to do the opposite: our goal is to select at least one variable from each group in the context of predictive regression modeling. This problem is NP-hard, but we propose the tightest convex relaxation: a composite penalty that is a combination of the 1 and 2 norms. Our so-called Exclusive Lasso method performs structured variable selection by ensuring that at least one variable is selected from each group. We study our method's statistical properties and develop computationally scalable algorithms for fitting the Exclusive Lasso. We study the effectiveness of our method via simulations as well as using NMR spectroscopy data. Here, we use the Exclusive Lasso to select the appropriate chemical shift from a dictionary of possible chemical shifts for each molecule in the biological sample.

show abstract

A totally unimodular view of structured sparsity

Cited by 2 publications

References 0 publications

The LASSO with Non-linear Measurements is Equivalent to One With Linear Measurements

The LASSO with Non-linear Measurements is Equivalent to One With Linear Measurements

Within group variable selection through the Exclusive Lasso

Contact Info

Product

Resources

About