How Correlations Influence Lasso Prediction

Hebiri, Mohamed; Lederer, Johannes

doi:10.1109/tit.2012.2227680

Cited by 95 publications

(80 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Least absolute shrinkage and selection operator regression provided a robust approach for specifying a predictive model from many candidate explanatory variables. Least absolute shrinkage and selection operator is known to be effective when explanatory variables are correlated (Dormann et al, 2013;Hebiri & Lederer, 2013), and so is particularly well-suited to the present data set, in which a few pairs of variables (e.g. TP and TN, TN and DOC, and TN and chl a) are strongly correlated (r > 0.7).…”

Section: Discussionmentioning

confidence: 99%

Managing microcystin: identifying national‐scale thresholds for total nitrogen and chlorophyll a

et al. 2014

View full text Add to dashboard Cite

1. The occurrence of high cyanotoxin concentrations can severely impair the use of a waterbody for drinking water and recreational purposes. Cyanotoxins are likely to occur under specific environmental conditions, and so identifying these conditions can facilitate management of the waterbody to reduce the likelihood of high cyanotoxin concentrations. 2. We analysed data collected from lakes across the contiguous United States to identify environmental variables that are strongly associated with occurrence of high concentrations of a common cyanotoxin, microcystin (MC). 3. Since many different environmental variables covary and are associated with high MC, we used least absolute shrinkage and selection operator (LASSO) regression to identify a few variables that provided accurate predictions of high MC (≥1 lg L À1 ). 4. Our analysis indicated that total nitrogen (TN) and chlorophyll a (chl a) concentrations yielded a parsimonious model that accurately predicted the occurrence of high MC. Based on this model, we identified management thresholds for TN and chl a that would maintain the probability of high MC at or below 10 and 5%.

show abstract

Section: Discussionmentioning

confidence: 99%

Managing microcystin: identifying national‐scale thresholds for total nitrogen and chlorophyll a

et al. 2014

View full text Add to dashboard Cite

show abstract

“…It has been discussed that adaptive lasso as a regularization approach gives more stable coefficient estimates in multicollinearity scenarios than those from OLS or ordinary lasso (Tibshirani, 1996; Zou, 2006; Hebiri and Lederer, 2013), as sparsity is imposed at different levels on each neighborhood. In contrast, for methods like PC-stable and MMHC, sparsity is utilized for the partial correlations as a whole view.…”

Section: Two Stage Solution Search Algorithmmentioning

confidence: 99%

“…On the other hand, the NS-DIST method may avoid those issues by employing an adaptive lasso score function. It has been discussed that adaptive lasso as a regularization approach gives more stable coefficient estimates in multicollinearity scenarios than those from OLS or ordinary lasso (Tibshirani, 1996; Zou, 2006; Hebiri and Lederer, 2013), as sparsity is imposed at different levels on each neighborhood.…”

Section: Simulation Studymentioning

confidence: 99%

Estimation of Directed Acyclic Graphs Through Two-Stage Adaptive Lasso for Gene Network Inference

Han

Chen²,

Cheon

et al. 2016

Journal of the American Statistical Association

View full text Add to dashboard Cite

Graphical models are a popular approach to find dependence and conditional independence relationships between gene expressions. Directed acyclic graphs (DAGs) are a special class of directed graphical models, where all the edges are directed edges and contain no directed cycles. The DAGs are well known models for discovering causal relationships between genes in gene regulatory networks. However, estimating DAGs without assuming known ordering is challenging due to high dimensionality, the acyclic constraints, and the presence of equivalence class from observational data. To overcome these challenges, we propose a two-stage adaptive Lasso approach, called NS-DIST, which performs neighborhood selection (NS) in stage 1, and then estimates DAGs by the Discrete Improving Search with Tabu (DIST) algorithm within the selected neighborhood. Simulation studies are presented to demonstrate the effectiveness of the method and its computational efficiency. Two real data examples are used to demonstrate the practical usage of our method for gene regulatory network inference.

show abstract

“…Recent work of Hebiri and Lederer (2013), Dalalyan et al (2014), and others has examined in some detail how correlations among the columns x ·j of the design matrix X affect the performance of the LASSO and the selection of the penalty parameter λ. The broad conclusion of this work is that smaller values of λ are appropriate when the columns of X are more (positively) correlated, a relationship that holds for the permutation selection procedure.…”

Section: Some Connections With Universal Thresholdsmentioning

confidence: 99%

A Permutation Approach for Selecting the Penalty Parameter in Penalized Model Selection

2015

View full text Add to dashboard Cite

Summary We describe a simple, computationally effcient, permutation-based procedure for selecting the penalty parameter in LASSO penalized regression. The procedure, permutation selection, is intended for applications where variable selection is the primary focus, and can be applied in a variety of structural settings, including that of generalized linear models. We briefly discuss connections between permutation selection and existing theory for the LASSO. In addition, we present a simulation study and an analysis of real biomedical data sets in which permutation selection is compared with selection based on the following: cross-validation (CV), the Bayesian information criterion (BIC), Scaled Sparse Linear Regression, and a selection method based on recently developed testing procedures for the LASSO.

show abstract

How Correlations Influence Lasso Prediction

Cited by 95 publications

References 23 publications

Managing microcystin: identifying national‐scale thresholds for total nitrogen and chlorophyll a

Managing microcystin: identifying national‐scale thresholds for total nitrogen and chlorophyll a

Estimation of Directed Acyclic Graphs Through Two-Stage Adaptive Lasso for Gene Network Inference

A Permutation Approach for Selecting the Penalty Parameter in Penalized Model Selection

Contact Info

Product

Resources

About