Variable Selection With Prior Information for Generalized Linear Models via the Prior LASSO Method

Abstract

LASSO is a popular statistical tool often used in conjunction with generalized linear models that can simultaneously select variables and estimate parameters. When there are many variables of interest, as in current biological and biomedical studies, the power of LASSO can be limited. Fortunately, so much biological and biomedical data have been collected and they may contain useful information about the importance of certain variables. This paper proposes an extension of LASSO, namely, prior LASSO (pLASSO), to incorporate that prior information into penalized generalized linear models. The goal is achieved by adding in the LASSO criterion function an additional measure of the discrepancy between the prior information and the model. For linear regression, the whole solution path of the pLASSO estimator can be found with a procedure similar to the Least Angle Regression (LARS). Asymptotic theories and simulation results show that pLASSO provides significant improvement over LASSO when the prior information is relatively accurate. When the prior information is less reliable, pLASSO shows great robustness to the misspecification. We illustrate the application of pLASSO using a real data set from a genome-wide association study.

Learning from a lot: Empirical Bayes for high‐dimensional model‐based prediction

“…These are generally computationally very demanding for typical high-dimensional settings with a large number of variables. Moreover, frequentist solutions have been proposed, which usually require additional tuning parameter(s) to cross-validate (Bergersen, Glad, & Lyng, 2011;Jiang, He, & Zhang, 2016b) or a group penalty (Meier, van de Geer, & Bühlmann, 2008;Simon, Friedman, Hastie, & Tibshirani, 2013). The latter may perform less well than EB-based regularization per group when the number of groups is small (Novianti et al, 2017).…”

Conditional screening for ultra-high dimensional covariates with survival outcomes

“…There has been some recent work on how to incorporate prior information. For example, Wang et al (2013) developed a LASSO method by assigning different prior distributions to each subset according to a modified Bayesian information criterion that incorporates prior knowledge on both the network structure and the pathway information, and Jiang et al (2015) proposed “prior lasso” (plasso) to balance between the prior information and the data. A natural extension of the current work is to develop a variable screening approach that incorporates more complex prior knowledge, such as the network structure or the spatial information of the covariates.…”

Building generalized linear models with ultrahigh dimensional features: A sequentially conditional approach

“…Condition (C) is justified by Jiang and Zhang () and Jiang et al . () and is similar to condition 3 in Bradic et al . ().…”

“…In practice, data are often standardized at the preprocessing stage, which may warrant the reasonableness of this condition. Condition (C) is justified by Jiang and Zhang (2013) and Jiang et al (2016) and is similar to condition 3 in Bradic et al (2011). The condition ensures the light tail of the response variable Y and is satisfied by a wide range of outcome data, including Gaussian and discrete data (such as binary and count data).…”