No abstract

No abstract

Minimax mean-squared error estimates of quadratic functionals of smooth functions have been constructed for a variety of smoothness classes. In contrast to many nonparametric function estimation problems there are both regular and irregular cases. In the regular cases the minimax mean-squared error converges at a rate proportional to the inverse of the sample size, whereas in the irregular case much slower rates are the rule.We investigate the problem of adaptive estimation of a quadratic functional of a smooth function when the degree of smoothness of the underlying function is not known. It is shown that estimators cannot achieve the minimax rates of convergence simultaneously over two parameter spaces when at least one of these spaces corresponds to the irregular case. A lower bound for the mean squared error is given which shows that any adaptive estimator which is rate optimal for the regular case must lose a logarithmic factor in the irregular case. On the other hand, we give a rather simple adaptive estimator which is sharp for the regular case and attains this lower bound in the irregular case. Moreover, we explicitly describe a subset of functions where our adaptive estimator loses the logarithmic factor and show that this subset is relatively small.

Estimation of the density of regression errors is a fundamental issue in regression analysis and it is typically explored via a parametric approach. This article uses a nonparametric approach with the mean integrated squared error (MISE) criterion. It solves a long-standing problem, formulated two decades ago by Mark Pinsker, about estimation of a nonparametric error density in a nonparametric regression setting with the accuracy of an oracle that knows the underlying regression errors. The solution implies that, under a mild assumption on the differentiability of the design density and regression function, the MISE of a data-driven error density estimator attains minimax rates and sharp constants known for the case of directly observed regression errors. The result holds for error densities with finite and infinite supports. Some extensions of this result for more general heteroscedastic models with possibly dependent errors and predictors are also obtained; in the latter case the marginal error density is estimated. In all considered cases a blockwise-shrinking Efromovich--Pinsker density estimate, based on plugged-in residuals, is used. The obtained results imply a theoretical justification of a customary practice in applied regression analysis to consider residuals as proxies for underlying regression errors. Numerical and real examples are presented and discussed, and the S-PLUS software is available.Comment: Published at http://dx.doi.org/10.1214/009053605000000435 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

The paper considers, for the first time in the literature, sharp minimax design of predictors and sharp minimax sequential estimation of regression functions in a classical heteroscedastic nonparametric regression. The suggested methodology of a sharp minimax design of predictors in controlled regression experiments with fixed-size samples is based on minimization of the coefficient of difficulty of an underlying regression model, which is defined as a factor in changing the sample size that makes the estimation problem comparable with estimation in a homoscedastic regression with unit-variance errors and uniform design. It is established that an optimal design density is proportional to an underlying scale function. This makes a sequential design of predictors, based on a sequential choice of design densities, a particularly attractive statistical strategy that allows the statistician to minimize the coefficient of difficulty during a controlled experiment. It is shown that a developed sequential design of predictors asymptotically matches the mean integrated squared error (MISE) of an oracle that knows the optimal design (the scale function). Another considered problem is a sequential estimation of an underlying regression function with a preassigned MISE and sharp minimax mean stopping time in a heteroscedastic regression setting where predictors are generated according to some (in general unknown) underlying design density. For this setting the theory of a sharp minimax sequential estimation of a regression function is developed and a sharp minimax sequential estimator is suggested. Finally, a problem of sequential estimation of a regression function based on a sequential design is considered. A procedure of sequential analysis of heteroscedastic regression is suggested that matches the performance of an oracle that knows the optimal design density and optimal stopping time, which implies a preassigned MISE of estimation of an underlying regression function. A discussion of possible extensions and further developments is also presented.