Prediction error bounds for linear regression with the TREX

Bien, Jacob; Gaynanova, Irina; Lederer, Johannes; Müller, Christian L.

doi:10.1007/s11749-018-0584-4

Cited by 19 publications

(10 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This bound parallels the one for the trex for 1 -regularized linear regression (Bien et al 2018b, Theorem 2). Moreover, it relates to Theorem 3.1; in particular, if λ ≤ λ * , then the t-ridge bound equals the edr bound at the optimal tuning parameter λ * -without the t-ridge knowing that tuning parameter.…”

Section: Further Theoretical Insightssupporting

confidence: 70%

Tuning-free ridge estimators for high-dimensional generalized linear models

Huang

Xie

Lederer

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Ridge estimators regularize the squared Euclidean lengths of parameters. Such estimators are mathematically and computationally attractive but involve tuning parameters that can be difficult to calibrate. In this paper, we show that ridge estimators can be modified such that tuning parameters can be avoided altogether. We also show that these modified versions can improve on the empirical prediction accuracies of standard ridge estimators combined with cross-validation, and we provide first theoretical guarantees.

show abstract

Section: Further Theoretical Insightssupporting

confidence: 70%

Tuning-free ridge estimators for high-dimensional generalized linear models

Huang

Xie

Lederer

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Bien et al [2016] showed the remarkable result that despite the nonconvexity, there exists a polynomial-time algorithm that is guaranteed to find the global minimum of the TREX problem. Bien et al [2018] recently established a prediction error bound for TREX, which deepens the understanding of the theoretical properties of TREX.…”

Section: Trexmentioning

confidence: 99%

A Survey of Tuning Parameter Selection for High-Dimensional Regression

Wang

2020

Annu. Rev. Stat. Appl.

View full text Add to dashboard Cite

Penalized (or regularized) regression, as represented by Lasso and its variants, has become a standard technique for analyzing highdimensional data when the number of variables substantially exceeds the sample size. The performance of penalized regression relies crucially on the choice of the tuning parameter, which determines the amount of regularization and hence the sparsity level of the fitted model. The optimal choice of tuning parameter depends on both the structure of the design matrix and the unknown random error distribution (variance, tail behavior, etc). This article reviews the current literature of tuning parameter selection for high-dimensional regression from both theoretical and practical perspectives. We discuss various strategies that choose the tuning parameter to achieve prediction accuracy or support recovery. We also review several recently proposed methods for tuning-free high-dimensional regression. arXiv:1908.03669v1 [stat.ME]

show abstract

“…In contrast, we derive both fast-rate and slow-rate bounds. The main advantage of the slow-rate bounds is that they do not require either sparsity assumption or the restricted eigenvalue condition, we refer to [2,9] for the discussion of the two types of bounds. As part of the slow-rate bound derivation, we demonstrate that the norm of B can 2 always be bounded by a constant times the norm of B * .…”

Section: Relations To Existing Literaturementioning

confidence: 99%

“…Proof of Lemma 1. Let f = f (Θ, B) denote the objective function in (2) and let (Θ * , B * ) be the global solution to (2), that is…”

Section: A1 Technical Proofsmentioning

confidence: 99%

Prediction and estimation consistency of sparse multi-class penalized optimal scoring

Gaynanova¹

2020

Bernoulli

Self Cite

View full text Add to dashboard Cite

Sparse linear discriminant analysis via penalized optimal scoring is a successful tool for classification in high-dimensional settings. While the variable selection consistency of sparse optimal scoring has been established, the corresponding prediction and estimation consistency results have been lacking. We bridge this gap by providing probabilistic bounds on out-of-sample prediction error and estimation error of multi-class penalized optimal scoring allowing for diverging number of classes.

show abstract

Prediction error bounds for linear regression with the TREX

Cited by 19 publications

References 46 publications

Tuning-free ridge estimators for high-dimensional generalized linear models

Tuning-free ridge estimators for high-dimensional generalized linear models

A Survey of Tuning Parameter Selection for High-Dimensional Regression

Prediction and estimation consistency of sparse multi-class penalized optimal scoring

Contact Info

Product

Resources

About