This paper is a selective review of the regularization methods scattered in statistics literature. We introduce a general conceptual approach to regularization and fit most existing methods into it. We have tried to focus on the importance of regularization when dealing with today's high-dimensional objects: data and models. A wide range of examples are discussed, including nonparametric regression, boosting, covariance matrix estimation, principal component estimation, subsampling.

No abstract

Asymptotic distributions, consistency, convergence rate, grouped data, imputation, iterative estimation, linear models, 62F12,

We present in this paper iterative estimation procedures, using conditional expectations, to fit linear models when the distributions of the errors are general and the dependent data stem from a finite number of sources, either grouped or non-grouped with different classification criteria. We propose an initial procedure that is inspired by the expectation-maximization (EM) algorithm, although it does not agree with it. The proposed procedure avoids the nested iteration, which implicitly appears in the initial procedure and also in the EM algorithm. The stochastic asymptotic properties of the corresponding estimators are analysed. Copyright 2004 Board of the Foundation of the Scandinavian Journal of Statistics..