Lasso penalized model selection criteria for high-dimensional multivariate linear regression analysis

Katayama, Shota; Imori, Shinpei

doi:10.1016/j.jmva.2014.08.002

Cited by 10 publications

(8 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When p n is greater than n, we cannot directly calculate S −1 and thus GC p . Therefore, we need different approaches to estimate a covariance matrix Σ such as sparse or ridge estimation (e.g., Yamamura, Yanagihara and Srivastava, 2010;Katayama and Imori, 2014;Fujikoshi and Sakurai, 2016). If we can estimate Σ accurately via these procedures, ALE and AME can be established by using it in place of S. It should also be noted that our proof depends on the assumption that the response matrix follows a Gaussian distribution.…”

Section: Discussionmentioning

confidence: 99%

“…The reason why this inversion arises may be that a difference in risks between two over-specified models (i.e., models including the true model) diverges with n and p n tending to infinity, and thus penalty terms of C p and AIC are moderate but that of BIC is too strong. In addition to these studies, model selection criteria in highdimensional data contexts and their consistency properties have been vigorously studied in various models and situations (e.g., Katayama and Imori, 2014;Imori and von Rosen, 2015;Yanagihara, 2015;Fujikoshi and Sakurai, 2016;Bai, Choi and Fujikoshi, 2018).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Asymptotic Optimality of Cp-Type Criteria in High-Dimensional Multivariate Linear Regression Models

Imori¹

2023

STAT SINICA

Self Cite

View full text Add to dashboard Cite

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Asymptotic Optimality of Cp-Type Criteria in High-Dimensional Multivariate Linear Regression Models

Imori¹

2023

STAT SINICA

Self Cite

View full text Add to dashboard Cite

“…It is straightforward to show that this is weaker than assuming lim inf n!y n À1 l min ðX 0 XÞ > 0, where l min ðAÞ is the minimum eigenvalue of a symmetric matrix A. The assumption for the true regression coe‰cients is essentially used in Katayama and Imori (2014). For example, when all the elements of each y l are nonzero constants not converging to 0, the assumption for the true regression coe‰cients holds.…”

Section: Assumptions For Consistencymentioning

confidence: 99%

“…Relaxing the normality assumption, Yanagihara (2019) focused on conditions for consistency of the GIC and GC p criterion under the hybridmoderate-high-dimensional asymptotic framework. As such, therein, p does not exceed n. On the other hand, in the context where p > n, Katayama and Imori (2014) considered variable selection criteria based on a lasso-type estimation for the inverse of the covariance matrix. Under the normality assumption, they showed that the criteria are consistent in a restricted-ultrahigh-dimensional asymptotic framework such that both n and p go to infinity but p may exceed n and log p=n !…”

Section: Introductionmentioning

confidence: 99%

Consistent variable selection criteria in multivariate linear regression even when dimension exceeds sample size

Oda¹

2020

Hiroshima Math. J.

View full text Add to dashboard Cite

This paper is concerned with the selection of explanatory variables in multivariate linear regression. The Akaike's information criterion and the C p criterion cannot perform in high-dimensional situations such that the dimension of a vector stacked with response variables exceeds the sample size. To overcome this, we consider two variable selection criteria based on an L 2 squared distance with a weighted matrix, namely the scalar-type generalized C p criterion and the ridge-type generalized C p criterion. We clarify conditions for their consistency under a hybrid-ultra-highdimensional asymptotic framework such that the sample size always goes to infinity but the number of response variables may not go to infinity. Numerical experiments show that the probabilities of selecting the true subset by criteria satisfying consistency conditions are high even when the dimension is larger than the sample size. Finally, we illuminate the practical utility of these criteria using empirical data.

show abstract

“…GIC is a popular procedure in choosing the final model in variable selection. In the literature there are many papers investigating its properties, for instance [18,32] in linear models, [7,14] in GLM, [16] in multivariate linear regression, [38] for SVM and [17] for general convex loss functions. GIC is often applied to pathwise algorithms under a common assumption that the true model is on this path.…”

Section: Introductionmentioning

confidence: 99%

Improving Lasso for model selection and prediction

Pokarowski¹,

Rejchel²,

Sołtys³

et al. 2019

Preprint

View full text Add to dashboard Cite

It is known that the Thresholded Lasso (TL), SCAD or MCP correct intrinsic estimation bias of the Lasso. In this paper we propose an alternative method of improving the Lasso for predictive models with general convex loss functions which encompass normal linear models, logistic regression, quantile regression or support vector machines. For a given penalty we order the absolute values of the Lasso non-zero coefficients and then select the final model from a small nested family by the Generalized Information Criterion. We derive exponential upper bounds on the selection error of the method. These results confirm that, at least for normal linear models, our algorithm seems to be the benchmark for the theory of model selection as it is constructive, computationally efficient and leads to consistent model selection under weak assumptions. Constructivity of the algorithm means that, in contrast to the TL, SCAD or MCP, consistent selection does not rely on the unknown parameters as the cone invertibility factor. Instead, our algorithm only needs the sample size, the number of predictors and an upper bound on the noise parameter. We show in numerical experiments on synthetic and real-world data sets that an implementation of our algorithm is more accurate than implementations of studied concave regularizations. Our procedure is contained in the R package ,,DMRnet" and available on the CRAN repository.

show abstract

Lasso penalized model selection criteria for high-dimensional multivariate linear regression analysis

Cited by 10 publications

References 22 publications

Asymptotic Optimality of Cp-Type Criteria in High-Dimensional Multivariate Linear Regression Models

Asymptotic Optimality of Cp-Type Criteria in High-Dimensional Multivariate Linear Regression Models

Consistent variable selection criteria in multivariate linear regression even when dimension exceeds sample size

Improving Lasso for model selection and prediction

Contact Info

Product

Resources

About