Non-asymptotic oracle inequalities for the Lasso and Group Lasso in high dimensional logistic model

Kwemou, Marius

doi:10.1051/ps/2015020

Cited by 16 publications

(18 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Applying Bach's (2010) Lemma 1 to each observation, as in Belloni, Chernozhukov, and Wei (2013) Kwemou (2012) also applied Bach's (2010) to study sparse logistic regression…”

Section: Supplemental Appendix For "Robust Inference On Average Treatmentioning

confidence: 99%

“…Bach (2010) only gives an error bound on coefficients in exactly sparse logistic regression, which can not yield our results; and does not consider prediction error or post-selection estimation. In independent work,(Kwemou 2012) andBelloni, Chernozhukov, and Wei (2013) also apply Bach's (2010) tools, but are focused on a different goals Vincent and Hansen (2014). apply the group lasso to multinomial logistic regression, but do not derive any theoretical results.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Robust inference on average treatment effects with possibly more covariates than observations

Farrell

2015

Journal of Econometrics

218

142

View full text Add to dashboard Cite

This paper concerns robust inference on average treatment effects following model selection. Under selection on observables, we construct confidence intervals using a doubly-robust estimator that are robust to model selection errors and prove their uniform validity over a large class of models that allows for multivalued treatments with heterogeneous effects and selection amongst (possibly) more covariates than observations. The semiparametric efficiency bound is attained under appropriate conditions. Precise conditions are given for any model selector to yield these results, and we specifically propose the group lasso, which is apt for treatment effects, and derive new results for high-dimensional, sparse multinomial logistic regression. Both a simulation study and revisiting the National Supported Work demonstration show our estimator performs well in finite samples.

show abstract

“…Applying Bach's (2010) Lemma 1 to each observation, as in Belloni, Chernozhukov, and Wei (2013) Kwemou (2012) also applied Bach's (2010) to study sparse logistic regression…”

Section: Supplemental Appendix For "Robust Inference On Average Treatmentioning

confidence: 99%

mentioning

confidence: 99%

Robust inference on average treatment effects with possibly more covariates than observations

Farrell

2015

Journal of Econometrics

218

142

View full text Add to dashboard Cite

show abstract

“…In linear models, these results include Bunea et al (2007) for prediction error of LASSO, Bickel et al (2009) for prediction error of both LASSO and Dantzig selector as well as bounds on estimation error, and Koltchinskii et al (2011) for a sharp oracle inequality of prediction error. Some parallel results have been developed for generalized linear models, including but not limited to, Van de Geer (2008), Bach (2010) and Kwemou (2012). In this subsection, we establish the oracle inequalities for pLASSO in the framework of generalized linear models, including its excess risk and estimation error.…”

Section: Theoretical Propertiesmentioning

confidence: 99%

“…The first part of Condition E was employed by Kwemou (2012) when they derived oracle inequalities for LASSO. The second part of Condition E, justified by Jiang and Zhang (2013), is used here to control the probability with which the oracle inequalities hold, by applying Bernstein’s inequality [e.g., Lemma 2.2.11 in van der Vaart and Wellner (1996)].…”

Section: Theoretical Propertiesmentioning

confidence: 99%

Variable Selection With Prior Information for Generalized Linear Models via the Prior LASSO Method

Jiang¹,

He²,

Zhang³

2016

Journal of the American Statistical Association

View full text Add to dashboard Cite

LASSO is a popular statistical tool often used in conjunction with generalized linear models that can simultaneously select variables and estimate parameters. When there are many variables of interest, as in current biological and biomedical studies, the power of LASSO can be limited. Fortunately, so much biological and biomedical data have been collected and they may contain useful information about the importance of certain variables. This paper proposes an extension of LASSO, namely, prior LASSO (pLASSO), to incorporate that prior information into penalized generalized linear models. The goal is achieved by adding in the LASSO criterion function an additional measure of the discrepancy between the prior information and the model. For linear regression, the whole solution path of the pLASSO estimator can be found with a procedure similar to the Least Angle Regression (LARS). Asymptotic theories and simulation results show that pLASSO provides significant improvement over LASSO when the prior information is relatively accurate. When the prior information is less reliable, pLASSO shows great robustness to the misspecification. We illustrate the application of pLASSO using a real data set from a genome-wide association study.

show abstract

“…This condition is weaker than Assumption D in Cheng et al (2016), which requires ρ p O n n log = ( log ) 1 5 ∕ . Condition (B) has been commonly assumed in the literature for variable selection and screening (Zhao and Li, 2012;Kwemou, 2016;Li et al, 2016). The uniform boundedness of X is adopted to simplify our theoretical development and can be relaxed to Conditions (B) and (D) in Fan and Song (2010).…”

Section: Introductionmentioning

confidence: 99%

Building generalized linear models with ultrahigh dimensional features: A sequentially conditional approach

Zheng

Hong

2019

Biometrics

View full text Add to dashboard Cite

Conditional screening approaches have emerged as a powerful alternative to the commonly used marginal screening, as they can identify marginally weak but conditionally important variables. However, most existing conditional screening methods need to fix the initial conditioning set, which may determine the ultimately selected variables. If the conditioning set is not properly chosen, the methods may produce false negatives and positives. Moreover, screening approaches typically need to involve tuning parameters and extra modeling steps in order to reach a final model. We propose a sequential conditioning approach by dynamically updating the conditioning set with an iterative selection process. We provide its theoretical properties under the framework of generalized linear models. Powered by an extended Bayesian information criterion as the stopping rule, the method will lead to a final model without the need to choose tuning parameters or threshold parameters. The practical utility of the proposed method is examined via extensive simulations and analysis of a real clinical study on predicting multiple myeloma patients’ response to treatment based on their genomic profiles.

show abstract

Non-asymptotic oracle inequalities for the Lasso and Group Lasso in high dimensional logistic model

Cited by 16 publications

References 35 publications

Robust inference on average treatment effects with possibly more covariates than observations

Robust inference on average treatment effects with possibly more covariates than observations

Variable Selection With Prior Information for Generalized Linear Models via the Prior LASSO Method

Building generalized linear models with ultrahigh dimensional features: A sequentially conditional approach

Contact Info

Product

Resources

About