Comparisons of penalized least squares methods by simulations

Abstract

Penalized least squares methods are commonly used for simultaneous estimation and variable selection in high-dimensional linear models. In this paper we compare several prevailing methods including the lasso, nonnegative garrote, and SCAD in this area through Monte Carlo simulations. Criterion for evaluating these methods in terms of variable selection and estimation are presented. This paper focuses on the traditional n > p cases.For larger p, our results are still helpful to practitioners after the dimensionality is reduced by a screening method.