A Plug-In Averaging Estimator for Regressions with Heteroskedastic Errors

Abstract

This paper proposes a novel model averaging estimator for the linear regression model with heteroskedastic errors. Unlike model selection which picks the single model among the candidate models, model averaging, on the other hand, incorporates all the information by averaging over all potential models. The two main questions of concern are: (1) How do we assign the weights for candidate models? (2) What is the asymptotic distribution of the averaging estimator and how do we make inference? This paper seeks to tackle these two problems from a frequentist view. First, we derive the asymptotic distribution of the averaging estimator with fixed weights in a local asymptotic framework. The optimal weights are obtained by minimizing the asymptotic mean-squared error (AMSE) of the averaging estimator. Second, we propose a plug-in averaging estimator which selects the weights by minimizing the sample analog of the AMSE. The asymptotic distribution of the proposed estimator is derived. Third, we show that the confidence intervals based on normal approximations suffer from size distortions. We suggest a plug-in method to construct the confidence interval which has good finite-sample coverage probability. The simulation results show that the plug-in averaging estimator performs favorably compared with other existing model selection and model averaging methods. As an empirical illustration, the proposed methodology is applied to estimate the effect of the student-teacher ratio on student achievement. We find that the insignificance of the student-teacher ratio variable from previous literature could be potentially explained by the fact of ignoring the model uncertainty.

Efficient shrinkage in parametric models

“…To gain insight, consider the canonical case W = V −1 , and write the distance statistic (7) as D n = pF n , where F n is an F-type statistic for (1). Using (18), this has the approximate expectation…”

“…We model the parameter vector as being in a n −1/2 -neighborhood of the specified restriction, so that the asymptotic distributions are continuous in the localizing parameter. This approach has been used successfully for averaging estimators by Hjort and Claeskens (2003) and Liu (2011), and for Stein-type estimators by Saleh (2006).…”

Jackknife model averaging for quantile regressions

“…The latter condition can be met in the case of nonparametric sieve estimation where   → ∞ as  → ∞ for  = 1   Alternatively, it is also automatically satisfied if one would like to consider the local asymptotic framework as in Hjort and Clasekens (2003), Leung and Barron (2006), Pötscher (2006), Clasekens and Hjort (2008), Hansen (2009Hansen ( , 2010, and Liu (2011) so that all models under consideration are asymptotically correctly specified. Note that these two conditions are not required for our JMA estimator.…”

“…In a local asymptotic framework Hjort and Clasekens (2003) and Clasekens and Hjort (2008, ch.7) study the asymptotic properties of the FMA maximum likelihood estimator by studying perturbations around a given narrow model in certain directions where the likelihood function is required to be second order continuously differentiable. Other works on the asymptotic property of averaging estimators include Leung and Barron (2006), Pötscher (2006), Hansen (2009Hansen ( , 2010, and Liu (2011). In particular, following Hjort and Clasekens (2003) and motivated by Hansen (2007), Liu (2011) derives the asymptotic distribution of the FMA estimator with fixed weights in a local asymptotic framework for linear regression models with heteroskedastic errors, and proposes a plug-in estimator of the optimal weights by minimizing the sample analog of the asymptotic mean squared error (MSE).…”

“…Other works on the asymptotic property of averaging estimators include Leung and Barron (2006), Pötscher (2006), Hansen (2009Hansen ( , 2010, and Liu (2011). In particular, following Hjort and Clasekens (2003) and motivated by Hansen (2007), Liu (2011) derives the asymptotic distribution of the FMA estimator with fixed weights in a local asymptotic framework for linear regression models with heteroskedastic errors, and proposes a plug-in estimator of the optimal weights by minimizing the sample analog of the asymptotic mean squared error (MSE). In a similar spirit, Liang, Zou, Wan, and Zhang (2011) derive an exact unbiased estimator of the MSE of the model average estimator and propose selecting the model weights that minimize the trace of the MSE estimate of focus parameters, but the weights are assumed to take a parametric functional form.…”