Estimation of the density of regression errors

Abstract

Estimation of the density of regression errors is a fundamental issue in regression analysis and it is typically explored via a parametric approach. This article uses a nonparametric approach with the mean integrated squared error (MISE) criterion. It solves a long-standing problem, formulated two decades ago by Mark Pinsker, about estimation of a nonparametric error density in a nonparametric regression setting with the accuracy of an oracle that knows the underlying regression errors. The solution implies that, under a mild assumption on the differentiability of the design density and regression function, the MISE of a data-driven error density estimator attains minimax rates and sharp constants known for the case of directly observed regression errors. The result holds for error densities with finite and infinite supports. Some extensions of this result for more general heteroscedastic models with possibly dependent errors and predictors are also obtained; in the latter case the marginal error density is estimated. In all considered cases a blockwise-shrinking Efromovich--Pinsker density estimate, based on plugged-in residuals, is used. The obtained results imply a theoretical justification of a customary practice in applied regression analysis to consider residuals as proxies for underlying regression errors. Numerical and real examples are presented and discussed, and the S-PLUS software is available.Comment: Published at http://dx.doi.org/10.1214/009053605000000435 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Estimation of the error density in a semiparametric transformation model

“…The estimator he proposed is constructed by splitting the sample into two parts: the first part is used for the estimation of the residuals, while the second part of the sample is used for the construction of the error density estimator. Efromovich (2005) proposed an adaptive estimator of the error density, based on a density estimator proposed by Pinsker (1980). Finally, Samb (2010) also considered the estimation of the error density, but his approach is more closely related to the one in Akritas and Van Keilegom (2001).…”

Bayesian bandwidth estimation for a nonparametric functional regression model with unknown error density

“…However, the estimation of error density is important to understand the residual behavior and to assess the adequacy of error distribution assumption (see for example, Akritas and Van Keilegom, 2001;Cheng and Sun, 2008); the estimation of error density is also useful to test the symmetry of the residual distribution (see for example, Ahmad and Li, 1997;Dette et al, 2002;Neumeyer and Dette, 2007); the estimation of error density is important to statistical inference, prediction and model validation (see for example, Efromovich, 2005;Muhsal and Neumeyer, 2010); and the estimation of error density is also useful for the estimation of the density of the response variable (see for example, Escanciano and Jacho-Chávez, 2012). In the realm of financial asset return, an important use of the estimated error density is to estimate value-at-risk for holding an asset.…”

Estimation of the density of regression errors by pointwise model selection

“…The reader is referred to Efromovich (2005) for practical applications. Many papers are devoted to density estimation but the difficulty in our problem is to estimate the density from a sample (ǫ i ) which is not observed.…”

“…Observing that ǫ i = Y i − b(X i ), we naturally estimate the errors by the residuals ( ǫ i = Y i − b(X i )), where b is an estimator of the regression function. Efromovich applies this strategy with a thresholding density estimation procedure (see for example Efromovich (2005)). He gets an estimator of the density of the (ǫ i ) whose L 2 -risk reaches the same minimax rate of convergence we would obtain if the (ǫ i ) were observed.…”