Gaussian model selection with an unknown variance

Abstract

Let $Y$ be a Gaussian vector whose components are independent with a common unknown variance. We consider the problem of estimating the mean $\mu$ of $Y$ by model selection. More precisely, we start with a collection $\mathcal{S}=\{S_m,m\in\mathcal{M}\}$ of linear subspaces of $\mathbb{R}^n$ and associate to each of these the least-squares estimator of $\mu$ on $S_m$. Then, we use a data driven penalized criterion in order to select one estimator among these. Our first objective is to analyze the performance of estimators associated to classical criteria such as FPE, AIC, BIC and AMDL. Our second objective is to propose better penalties that are versatile enough to take into account both the complexity of the collection $\mathcal{S}$ and the sample size. Then we apply those to solve various statistical problems such as variable selection, change point detections and signal estimation among others. Our results are based on a nonasymptotic risk bound with respect to the Euclidean loss for the selected estimator. Some analogous results are also established for the Kullback loss.Comment: Published in at http://dx.doi.org/10.1214/07-AOS573 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Segmentation of the mean of heteroscedastic data via cross-validation

“…Lebarbier (2005) proposed c 1 = 5 and c 2 = 2 for optimizing the penalty (2) in the context of change-point detection. Penalties similar to (2) have been introduced independently by other authors (Rissanen 1983;Abramovich et al 2006;Barron et al 1999;Tibshirani and Knight 1999;Baraud et al 2009) and are shown to provide satisfactory results. Nevertheless, all these results about exponential collections assume that data are homoscedastic.…”

“…BGH In the context of Gaussian regression, Baraud et al (2009) propose a new penalized criterion, which does not depend on the a priori knowledge of the variance of the noise: The selected model results from the minimization over m ∈ M n of the criterion crit BGH (m)…”

“…where pen BGH (m) is defined in Baraud et al (2009) and depends on a constant chosen by simulation to be equal to K = 1.1.…”

Penalized likelihood and multiple testing

“…We also remark here that Baraud, Giraud, and Huet (2009) study BIC along with FPE, AIC, and AMDL. However they do not study the procedures as multiple testing procedures.…”

Adaptive thresholding of sequences with locally variable strength

“…Alternative approaches to this problem also exist such as the model selection approach of (Baraud et al 2008) where a penalty criterion is introduced dependent upon the number of coefficients estimated as non-zero; or the work of (Beran 2004) where the penalized least squares (PLS) estimator are represented as hybrid shrinkage estimators.…”