A Dirichlet process functional approach to heteroscedastic-consistent covariance estimation

Abstract

The mixture of Dirichlet process (MDP) defines a flexible prior distribution on the space of probability measures. This study shows that ordinary least-squares (OLS) estimator, as a functional of the MDP posterior distribution, has posterior mean given by weighted least-squares (WLS), and has posterior covariance matrix given by the (weighted) heteroscedastic-consistent sandwich estimator. This is according to a pairs bootstrap distribution approximation of the posterior, using a Pólya urn scheme. Also, when the MDP prior baseline distribution is specified as a product of independent probability measures, this WLS solution provides a new type of generalized ridge regression estimator. Such an estimator can handle multicollinear or singular design matrices even when the number of covariates exceeds the sample size, and can shrinks the coefficient estimates of irrelevant covariates towards zero, which makes it useful for nonlinear regressions via basis expansions. Also, this MDP/OLS functional methodology can be extended to methods for analyzing the sensitivity of the heteroscedasticity-consistent causal effect size over a range of hidden biases due to missing covariates omitted from the regression, and more generally extended to a Vibration of Effects analysis. The methodology is illustrated through the analysis of simulated and real data sets. Overall, this study establishes new connections between Dirichlet process functional inference, the bootstrap, consistent sandwich covariance estimation, ridge shrinkage regression, WLS, and sensitivity analysis, to provide regression methodology useful for inferences of the mean dependent response.

Convergence behaviours of energy series and GDP nexus hypothesis: A non-parametric Bayesian application

“…The covariance matrix of OLS estimator for the regression coefficients appears to be inconsistent in the case of mis specified model mainly due to heteroscedasticity bias. One way to deal with this issue is to use White’s sandwich covariance matrix estimator [ 73 ]. Also, it is worth notifying that recent studies employed an alternative Bayesian approach that could be extended to include informative prior distributions.…”

“…Dependent Dirichlet Process (DDP) is employed in most of the Bayesian density regressions [ 73 ]. DDP prior is G x ~ DDP ( α , G 0 x ).…”

The Macroeconomy as a Random Forest

“…22 From a Bayesian point of view, Lancaster (2003) show that the obtained variance for OLS from using such a bootstrap is asymptotically equivalent to that of White's heteroscedasticity-robust sandwich formula. Poirier (2011) propose better priors and Karabatsos (2016) incorporate such ideas into a generalized ridge regression. Hence, in the spirit of heteroscedasticity-robust estimation, no attempt will be made at directly modeling stochastic volatility (which is a GLS approach) but it will rather be reflected in larger bands for periods of high volatility.…”