Sequential Design and Estimation in Heteroscedastic Nonparametric Regression

Abstract

The paper considers, for the first time in the literature, sharp minimax design of predictors and sharp minimax sequential estimation of regression functions in a classical heteroscedastic nonparametric regression. The suggested methodology of a sharp minimax design of predictors in controlled regression experiments with fixed-size samples is based on minimization of the coefficient of difficulty of an underlying regression model, which is defined as a factor in changing the sample size that makes the estimation problem comparable with estimation in a homoscedastic regression with unit-variance errors and uniform design. It is established that an optimal design density is proportional to an underlying scale function. This makes a sequential design of predictors, based on a sequential choice of design densities, a particularly attractive statistical strategy that allows the statistician to minimize the coefficient of difficulty during a controlled experiment. It is shown that a developed sequential design of predictors asymptotically matches the mean integrated squared error (MISE) of an oracle that knows the optimal design (the scale function). Another considered problem is a sequential estimation of an underlying regression function with a preassigned MISE and sharp minimax mean stopping time in a heteroscedastic regression setting where predictors are generated according to some (in general unknown) underlying design density. For this setting the theory of a sharp minimax sequential estimation of a regression function is developed and a sharp minimax sequential estimator is suggested. Finally, a problem of sequential estimation of a regression function based on a sequential design is considered. A procedure of sequential analysis of heteroscedastic regression is suggested that matches the performance of an oracle that knows the optimal design density and optimal stopping time, which implies a preassigned MISE of estimation of an underlying regression function. A discussion of possible extensions and further developments is also presented.

Optimal adaptive inference in random design binary regression

“…Binary regression automatically belongs to a heteroscedastic variance regime-a more challenging scenario in general , Efromovich and Pinsker (1996), Antoniadis, Besbeas and Sapatinas (2001), , Galtchouk and Pergamenshchikov (2008), Galtchouk and Pergamenshchikov (2009)). However, it is different from standard heteroscedastic additive Gaussian noise regression problems in that the mean regression function is intimately tied to the conditional variance and shares the same smoothness.…”

Minimax and bayes estimation in deconvolution problem

“…The presence of ||h|| 2 in the asymptotic Ψ (µ ) of asymptotically minimax risk ρ and in the definition (9) of asymptotically minimax estimator x * * µ is the unique difference from the standard definitions of asymptotically minimax risks and asymptotically minimax estimator (see [8][9][10][11]26]). Remark 1.…”

“…The kernel k µ is the kernel of standard asymptotically minimax estimator x * * µ (t) = (k µ * y)(t) (see [8][9][10][11]26]). The kernel k ω is the kernel of simple projection estimator x * * ω (t, y) = (k ω * y)(t).…”

Non-Parametric Sequential Estimation of a Regression Function Based on Dependent Observations

“…(2002); Martinsek (1992); Mukhopadhyay and Solanky (1994); Rao (1983); Stute (1983); Vasiliev (1997); Vasiliev and Koshkin (1998); Xu and Martinsek (1995)). Peculiarities and difficulties of sequential approach to nonparametric problems are discussed in detail in Efroimovich (2007).…”

“…Sequential approach has also been applied to non-parametric regression and density function estimation problems as well (see, e.g., Carroll (1976); Davies and Wegman (1975), Dharmasena at al. (2008)-Efroimovich (2007, Hondaa (1998); Liski at al. (2002); Martinsek (1992); Mukhopadhyay and Solanky (1994); Rao (1983); Stute (1983); Vasiliev (1997); Vasiliev and Koshkin (1998); Xu and Martinsek (1995)).…”