Model selection via testing: an alternative to (penalized) maximum likelihood estimators

Abstract

This paper is devoted to the description and study of a family of estimators, that we shall call T -estimators (T for tests), for minimax estimation and model selection. Their construction is based on former ideas about deriving estimators from some families of tests due to Le Cam (1973 and1975) and Birgé (1983, 1984a and and about complexity based model selection from Barron and Cover (1991).It is well-known that maximum likelihood estimators or, more generally, minimum contrast estimators do suffer from various weaknesses, and their penalized versions as well. In particular they are not robust and they require restrictive assumptions on both the models and the underlying parameter for the estimators to work correctly. Our method, which derives an estimator from many simultaneous tests between some probability balls in a suitable metric space, tends to solve many of these difficulties. Its robustness properties allow to deal with minimax estimation and model selection in a unified way, since bounding the minimax risk amounts to perform the method with a single, well-chosen, model. This results in simple bounds for the minimax risk solely based on some metric properties of the parameter space. Moreover the method applies to various statistical frameworks (we shall concentrate here on the i.i.d. and the Gaussian sequence settings only) and can handle essentially all types of models, linear or not, parametric and non-parametric, simultaneously. From these viewpoints, it is much more flexible than traditional methods. In particular, we shall be able to derive some adaptation results for density estimation over Besov balls that do not seem to be accessible to classical methods. The counterpart for these nice properties is the very high computational complexity of our construction which makes our estimators look more like theoretical than practical tools. 0 AMS 1991 subject classifications. Primary 62F35; secondary 62G05.

From Local Kernel to Nonlocal Multiple-Model Image Denoising

“…A ßow of publications on aggregation can be seen in mathematical statistics (e.g., Birge [8], Bunea et al [14], Goldenshluger [42]). The problems studied in these works are mainly concentrated on comparison and selection of the best estimator from a given set of estimators.…”

Model selection for Poisson processes

“…In order to explain our method of estimation and model selection, we need to recall some general results from Birgé [9] about T-estimators that we shall specialize to the specific framework of this paper. Let (M, d) be some metric space and B(t, r) denote the open ball of center t and radius r in M .…”

“…We shall also show that recent results by Rigollet and Tsybakov [32] on aggregation of estimators for density estimation extend straightforwardly to the Poisson case. In Section 3, we briefly recall the general construction of T-estimators introduced in Birgé [9] and apply it to the specific case of Poisson processes. We also provide an illustration based on nonlinear approximating models.…”

“…The problem of understanding to what extent (1.12) does hold has been treated in many papers using various methods, mostly based on the minimization of some penalized criterion. A special construction based on testing has been introduced in Birgé [9] and then applied to different stochastic frameworks. We shall show here that this construction also applies to Poisson processes and then derive the numerous consequences of this property.…”

Estimating a discrete distributionviahistogram selection

“…Moreover, the d-estimator can be implemented with only O(n) computations. Notice that a similar collection of linear spaces has lately been used by Birgé [5,6] and Baraud and Birgé [2] for estimation by model selection in various statistical frameworks.…”