Optimal two-step prediction in regression

Chételat, Didier; Lederer, Johannes; Salmon, Joseph

doi:10.1214/17-ejs1287

Cited by 19 publications

(9 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, our essay does not provide any guidance on how to select tuning parameters in practice; indeed, the tuning parameters in Lemma 2.1 depend on the noise ε, which is unknown in practice. For ideas on the practical selection of the lasso tuning parameter with finite sample guarantees, we refer to [14,15]. For ideas on how to make the selection of tuning parameters independent of unknown model aspects, we refer to [17,27] and the square-root/scaled lasso example in the following section.…”

Section: General Resultsmentioning

confidence: 99%

Oracle inequalities for high-dimensional prediction

Lederer¹,

Lu²,

Gaynanova³

2019

Bernoulli

Self Cite

View full text Add to dashboard Cite

The abundance of high-dimensional data in the modern sciences has generated tremendous interest in penalized estimators such as the lasso, scaled lasso, square-root lasso, elastic net, and many others. In this paper, we establish a general oracle inequality for prediction in high-dimensional linear regression with such methods. Since the proof relies only on convexity and continuity arguments, the result holds irrespective of the design matrix and applies to a wide range of penalized estimators. Overall, the bound demonstrates that generic estimators can provide consistent prediction with any design matrix. From a practical point of view, the bound can help to identify the potential of specific estimators, and they can help to get a sense of the prediction accuracy in a given application.

show abstract

Section: General Resultsmentioning

confidence: 99%

Oracle inequalities for high-dimensional prediction

Lederer¹,

Lu²,

Gaynanova³

2019

Bernoulli

Self Cite

View full text Add to dashboard Cite

show abstract

“…A second vein of literature concerns data-adaptive methods for the Lasso. Examples include (i) [40], who study the asymptotic properties of optimally tuned (with respect to mean square error) Lasso estimators in connection with solutions to adaptively tuned approximate message passing algorithms; (ii) AV ∞ of [21], who provide ℓ ∞ estimation error guarantees; (iii) AV Pr of [20], who provide guarantees for a post-lasso procedure under prediction error loss; (iv) stability selection for variable selection in [39] and subsequent work by [47]; (v) LinSelect of [7,28]; and (vi) [13], who cite the oracle bounds of [55,56,57] to establish the asymptotic guarantees in prediction loss for the cross-validated highly adaptive Lasso (HAL) estimator.…”

Section: Estimation Error Boundsmentioning

confidence: 99%

Inference for high-dimensional instrumental variables regression

Gold

Lederer

Tao

2020

Journal of Econometrics

Self Cite

View full text Add to dashboard Cite

This paper concerns statistical inference for the components of a high-dimensional regression parameter despite possible endogeneity of each regressor. Given a first-stage linear model for the endogenous regressors and a second-stage linear model for the response variable, we develop a novel adaptation of the parametric one-step update to a generic second-stage estimator. We provide high-level conditions under which the scaled update is asymptotically normal. We introduce a two-stage Lasso procedure and show that, under a sub-Gaussian noise regime, the second-stage Lasso estimator satisfies the aforementioned conditions. Using these results, we construct asymptotically valid confidence intervals for the components of the second-stage regression vector. We complement our asymptotic theory with empirical studies, which demonstrate the relevance of our method in finite samples.

show abstract

“…While this work is focused exclusively on the TREX, for completeness, we mention some alternative approaches to tuning parameter calibration. Calibration schemes for the LASSO have been introduced and studied in various papers, including [15,16,17,29,39,42,43]. A LASSO-type algorithm with variable selection guarantees was introduced in [37].…”

Section: Related Literaturementioning

confidence: 99%

Prediction error bounds for linear regression with the TREX

Bien

Gaynanova

Lederer

et al. 2018

TEST

Self Cite

View full text Add to dashboard Cite

The TREX is a recently introduced approach to sparse linear regression. In contrast to most well-known approaches to penalized regression, the TREX can be formulated without the use of tuning parameters. In this paper, we establish the first known prediction error bounds for the TREX. Additionally, we introduce extensions of the TREX to a more general class of penalties, and we provide a bound on the prediction error in this generalized setting. These results deepen the understanding of TREX from a theoretical perspective and provide new insights into penalized regression in general.

show abstract

Optimal two-step prediction in regression

Cited by 19 publications

References 39 publications

Oracle inequalities for high-dimensional prediction

Oracle inequalities for high-dimensional prediction

Inference for high-dimensional instrumental variables regression

Prediction error bounds for linear regression with the TREX

Contact Info

Product

Resources

About