On the Sensitivity of the Lasso to the Number of Predictor Variables

Flynn, Cheryl J.; Hurvich, Clifford M.; Simonoff, Jeffrey S.

doi:10.1214/16-sts586

Cited by 13 publications

(10 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although various loss bounds have been derived that support the use of the lasso for a deterministic choice of the regularization parameter, in practice, the tuning parameter is chosen data dependently with good reasons. As pointed out in the work of Flynn et al (2014), the loss of the lasso when using data-dependent tuning parameters and without knowing which variables have non-zero coefficients as compared to the loss obtained for the true sparse model is much larger than suggested by oracle inequalities. They demonstrate the effect for categorical predictors in a small simulation study.…”

Section: Discussion By J Chiquet Y Grandvalet and G Rigaillmentioning

confidence: 95%

Rejoinder: Regularized regression for categorical data

Tutz

Gertheiss

2016

Statistical Modelling

View full text Add to dashboard Cite

Section: Discussion By J Chiquet Y Grandvalet and G Rigaillmentioning

confidence: 95%

Rejoinder: Regularized regression for categorical data

Tutz

Gertheiss

2016

Statistical Modelling

View full text Add to dashboard Cite

“…Even though one of the analyzed data sets is identical in both studies, there are (minor) discrepancies between the results of the two studies due to differences in study design and implementation details. For example, it is well-known that the results of LASSO depend on the number of variables [39], so it is most likely that the pre-selection step (not performed in De Bin et al [6]) changes the results noticeably. Other discrepancies may be related to the use of a FP function to model age and to differences in the standardization step when performing boosting/lasso (in De Bin et al [6] the standardization was performed on the whole training set, while we standardized the variables on the single subsamples).…”

Section: Discussionmentioning

confidence: 99%

A plea for taking all available clinical information into account when assessing the predictive value of omics data

Volkmann

Bin

Sauerbrei

et al. 2019

BMC Med Res Methodol

View full text Add to dashboard Cite

Background Omics data can be very informative in survival analysis and may improve the prognostic ability of classical models based on clinical risk factors for various diseases, for example breast cancer. Recent research has focused on integrating omics and clinical data, yet has often ignored the need for appropriate model building for clinical variables. Medical literature on classical prognostic scores, as well as biostatistical literature on appropriate model selection strategies for low dimensional (clinical) data, are often ignored in the context of omics research. The goal of this paper is to fill this methodological gap by investigating the added predictive value of gene expression data for models using varying amounts of clinical information. Methods We analyze two data sets from the field of survival prognosis of breast cancer patients. First, we construct several proportional hazards prediction models using varying amounts of clinical information based on established medical knowledge. These models are then used as a starting point (i.e. included as a clinical offset) for identifying informative gene expression variables using resampling procedures and penalized regression approaches (model based boosting and the LASSO). In order to assess the added predictive value of the gene signatures, measures of prediction accuracy and separation are examined on a validation data set for the clinical models and the models that combine the two sources of information. Results For one data set, we do not find any substantial added predictive value of the omics data when compared to clinical models. On the second data set, we identify a noticeable added predictive value, however only for scenarios where little or no clinical information is included in the modeling process. We find that including more clinical information can lead to a smaller number of selected omics predictors. Conclusions New research using omics data should include all available established medical knowledge in order to allow an adequate evaluation of the added predictive value of omics data. Including all relevant clinical information in the analysis might also lead to more parsimonious models. The developed procedure to assess the predictive value of the omics data can be readily applied to other scenarios. Electronic supplementary material The online version of this article (10.1186/s12874-019-0802-0) contains supplementary material, which is available to authorized users.

show abstract

“…Formal proofs for the case of diagonal loading were provided by Pajovic [129,Chapter 3], wherein the optimum diagonal loading coefficient δ 2 is predicted using Random Matrix Theory and the sensitivity of the optimum diagonal loading to the physics of the problem is analyzed for array processing. Similarly, it has been shown [62] that when δ 1 of (2.16) is chosen in a data-dependent manner, the performance of the LASSO estimator can rapidly deteriorate in practice. Indeed, experience also bears this out, as a poor choice of parameter can cause regularized solution to perform worse than expected.…”

Section: The Need For Alternate Approachesmentioning

confidence: 99%

Graphical model driven methods in adaptive system identification

Yellepeddi¹

2016

View full text Add to dashboard Cite

Identifying and tracking an unknown linear system from observations of its inputs and outputs is a problem at the heart of many different applications. Due to the complexity and rapid variability of modern systems, there is extensive interest in solving the problem with as little data and computation as possible.This thesis introduces the novel approach of reducing problem dimension by exploiting statistical structure on the input. By modeling the input to the system of interest as a graph-structured random process, it is shown that a large parameter identification problem can be reduced into several smaller pieces, making the overall problem considerably simpler.Algorithms that can leverage this property in order to either improve the performance or reduce the computational complexity of the estimation problem are developed. The first of these, termed the graphical expectation-maximization least squares (GEM-LS) algorithm, can utilize the reduced dimensional problems induced by the structure to improve the accuracy of the system identification problem in the low sample regime over conventional methods for linear learning with limited data, including regularized least squares methods.Next, a relaxation of the GEM-LS algorithm termed the relaxed approximate graph structured least squares (RAGS-LS) algorithm is obtained that exploits structure to perform highly efficient estimation. The RAGS-LS algorithm is then recast into a recursive framework termed the relaxed approximate graph structured recursive least squares (RAGS-RLS) algorithm, which can be used to track time-varying linear systems with low complexity while achieving tracking performance comparable to much more computationally intensive methods.The performance of the algorithms developed in the thesis in applications such as channel identification, echo cancellation and adaptive equalization demonstrate that the gains admitted by the graph framework are realizable in practice. The methods have wide applicability, and in particular show promise as the estimation and adaptation algorithms for a new breed of fast, accurate underwater acoustic modems.The contributions of the thesis illustrate the power of graphical model structure in simplifying difficult learning problems, even when the target system is not directly structured.Thesis Supervisor: James C. Preisig Title: Scientist Emeritus, Woods Hole Oceanographic Institution AcknowledgmentsPiglet noticed that even though he had a Very Small Heart, it could hold a rather large amount of Gratitude. Winnie the PoohA.A.MilneMy years at MIT have been about so much more than a degree. In the past six years you have introduced to more worlds than I ever knew existed. As this long and wonderful journey ends, I find that my heart can, too, hold so much gratitude, for everything that you have given.Of course, I haven't said who "you" is, so here goes.It is impossible to write any list of acknowledgements without having at the very top my thesis advisor Dr. James Preisig, who has truly been a wide-angle lens f...

show abstract

On the Sensitivity of the Lasso to the Number of Predictor Variables

Cited by 13 publications

References 34 publications

Rejoinder: Regularized regression for categorical data

Rejoinder: Regularized regression for categorical data

A plea for taking all available clinical information into account when assessing the predictive value of omics data

Graphical model driven methods in adaptive system identification

Contact Info

Product

Resources

About