Improved minimax predictive densities under Kullback–Leibler loss

Abstract

Let X|µ ∼ Np(µ, vxI) and Y |µ ∼ Np(µ, vyI) be independent pdimensional multivariate normal vectors with common unknown mean µ. Based on only observing X = x, we consider the problem of obtaining a predictive densityp(y|x) for Y that is close to p(y|µ) as measured by expected Kullback-Leibler loss. A natural procedure for this problem is the (formal) Bayes predictive densitypU(y|x) under the uniform prior πU(µ) ≡ 1, which is best invariant and minimax. We show that any Bayes predictive density will be minimax if it is obtained by a prior yielding a marginal that is superharmonic or whose square root is superharmonic. This yields wide classes of minimax procedures that dominatepU(y|x), including Bayes predictive densities under superharmonic priors. Fundamental similarities and differences with the parallel theory of estimating a multivariate normal mean under quadratic loss are described.

Predictive Density Estimation for Multiple Regression

“…Our results can be seen as a substantial generalization of George, Liang and Xu (2006) who considered the special case of this problem when X ∼ N m (µ, σ 2 x I) and Y ∼ N m (µ, σ 2 y I), where µ is the common unknown multivariate normal mean. Moving further away from this common mean setup, we proceed in Section 3 to extend these results to the setting where only a subset of the p predictors is considered to be potentially irrelevant.…”

Singular value shrinkage priors for Bayesian prediction

“…Proposition 1 is without an assumption concerning twice differentiablility of π and is a slight generalization of the results in George et al (2006).…”

“…We extended the Bayesian shrinkage prediction method for vector-variate Normal distributions of Komaki (2001) and George et al (2006) to matrixvariate case. Then, we can consider further extensions to tensor-variate case.…”

“…Since the Stein prior puts more weight near the origin, the amount of risk reduction is large when the true value of µ is near the origin. George et al (2006) generalized this result and proved that Bayesian predictive densities based on superharmonic priors dominate those based on the Jeffreys prior under the Kullback-Leibler risk. Next, Kobayashi & Komaki (2008) and George & Xu (2008) considered the cases where Σ is not necessarily proportional to Σ. Bayesian predictive densities based on superharmonic priors dominate those based on the uniform prior under the Kullback-Leibler risk also in this general situation.…”

“…Here, we consider the case where Σ⊗ C is proportional to Σ ⊗ C. This assumption corresponds to the setting of Komaki (2001) and George et al (2006). The general covariance case is considered in Section 2.5.…”