Persistence in high-dimensional linear predictor selection and the virtue of overparametrization

ℓ 1-regularized linear regression: persistence and oracle inequalities

“…Motivated by many practical prediction problems, including those that arise in microarray data analysis and natural language processing, this problem has been extensively studied in recent years. The results can be divided into two categories: those that study the predictive power ofβ [9,30,12] and those that study its sparsity pattern and reconstruction properties [4,32,18,19,17,8]; this article falls into the first of these categories.…”

“…An outcome of these estimates is a solution to the question of persistence, posed by Greenshtein and Ritov [12], which is defined as follows. Let (d n ) ∞ n=1 be an increasing sequence, consider a sequence of measures (µ n ) ∞ n=1 on R dn × R and suppose that for every n, one is given n independent samples (X 1 , Y 1 ), .…”

Regularization in statistics

“…It is interesting to note that if we are interested in a classification goal such as implementing the Fisher linear discriminant function, then an alternative approach which consider classifiers, based on linear predictors as we discussed, without reference to an underlying distribution such as the Gaussian, then results comparable to Bickel and Levina (2004) have been obtained by Greenshtein and Ritov (2004) and Greenshtein (2006). We note also that there is an extensive literature on using Bayesian methods in estimation of Σ under parametric assumptions (Smith and Kohn, 2002).…”

“…The major work in the context of Goal (I) has been the work of Greenshtein and Ritov (2004), Greenshtein (2006), Meinshausen (2005), and to some extent in Bickel and Levina (2004). In the context of Goal (II), Fan and coworkers (Fan and Li, 2006, and refer-ences therein) have also looked at many generalizations of the regression model we have focussed on in the large n, p context.…”

Sliced Inverse Regression

“…The dimension p of the covariate was also comparably high. Greenshtain and Ritov (2004) studied the asymptotics of lasso methods when p goes to infinity as n tends to infinity. Bickel and Levina (2004) investigated the asymptotic properties for Fisher's linear discriminant analysis (for relevant data analysis see Levina 2002).…”