Tuning Parameter Selection in High Dimensional Penalized Likelihood

Abstract

Determining how to appropriately select the tuning parameter is essential in penalized likelihood methods for high-dimensional data analysis. We examine this problem in the setting of penalized likelihood methods for generalized linear models, where the dimensionality of covariates p is allowed to increase exponentially with the sample size n. We propose to select the tuning parameter by optimizing the generalized information criterion (GIC) with an appropriate model complexity penalty. To ensure that we consistently identify the true model, a range for the model complexity penalty is identified in GIC. We find that this model complexity penalty should diverge at the rate of some power of $\log p$ depending on the tail probability behavior of the response variables. This reveals that using the AIC or BIC to select the tuning parameter may not be adequate for consistently identifying the true model. Based on our theoretical study, we propose a uniform choice of the model complexity penalty and show that the proposed approach consistently identifies the true model among candidate models with asymptotic probability one. We justify the performance of the proposed procedure by numerical simulations and a gene expression data analysis.Comment: 38 page

Understanding the economic determinants of the severity of operational losses: A regularized generalized Pareto regression approach

“…These findings are consistent with previous works that emphasize the good performance of BIC over AIC [Fan andTang, 2012, Groll andTutz, 2014]. We see that, even in more realistic frameworks, this technique allows to exclude the non-informative covariates.…”

Big Data Analytics: A New Perspective

“…This has subsequently been generalised to the analysis of functions involving L q , 0 q 2; norms. While these techniques have found considerable use in econometrics, their theoretical properties have been mainly analysed in the statistical literature starting with the seminal work of Tibshirani (1996) and followed up with important contributions by Frank and Friedman (1993), Zhou and Hastie (2005), Lv and Fan (2009), Efron, Hastie, Johnstone, and 2 Tibshirani (2004), Bickel, Ritov, and Tsybakov (2009), Candes and Tao (2007), Zhang (2010), Fan and Li (2001), Antoniadis and Fan (2001), Fan and Lv (2013) and Fan and Tang (2013).…”

Adjusted Adaptive LASSO in High-dimensional Poisson Regression Model

“…4;2015 depends on the data, it can be computed using cross-validation method (Y. Fan & Tang, 2013) (James, Witten, Hastie, & Tibshirani, 2013). Before solving the PPR, it is worth to make centering to the y and standardization , this is to make the intercept ( 0 β ) equals zero.…”