Empirical Bayes predictive densities for high-dimensional normal models

Abstract

a b s t r a c tThis paper addresses the problem of estimating the density of a future outcome from a multivariate normal model. We propose a class of empirical Bayes predictive densities and evaluate their performances under the Kullback-Leibler (KL) divergence. We show that these empirical Bayes predictive densities dominate the Bayesian predictive density under the uniform prior and thus are minimax under some general conditions. We also establish the asymptotic optimality of these empirical Bayes predictive densities in infinitedimensional parameter spaces through an oracle inequality.

Exact minimax estimation of the predictive density in sparse Gaussian models

“…Decision theoretic parallels between predictive density estimation under Kullback–Leibler loss and point estimation under quadratic loss have been explored in our Gaussian model by George, Liang and Xu (2006), Ghosh, Mergel and Datta (2008), Komaki (2004), Xu and Zhou (2011) and George, Liang and Xu (2012). For unconstrained parameter spaces $\mathrm{normal\Theta }={\mathbb{ℝ}}^n$, fundamental ideas in Gaussian point estimation theory can be extended to yield optimal predictive density estimates [Brown, George and Xu (2008), Fourdrinier et al (2011), Komaki (2001)].…”

Optimal Shrinkage Estimation of Predictive Densities Under α-Divergences

“…The observed past X and unobserved future Y are only related through the unknown location parameter θ = {θ i : 1 ≤ i ≤ n}. This is the heteroskedastic version of the Gaussian predictive model studied in Komaki (2001); George et al (2006); Brown et al (2008); Xu and Zhou (2011). Letp(y|x) be any prde for the true density p(y|θ, ν)…”

“…Hereon, we describe our method first for the diagonal predictive set-up as it produces comparatively simpler expressions that can be intuitively studied and compared to the point estimation results of Xie, Kou, and Brown (2012) and the predictive KL results in Xu and Zhou (2011). The general results for non-diagonal set-ups along with their complete proofs are provided in George et al (2021, Section 1).…”

“…Dimension independent non-asymptotic characterizations of the predictive risk of our proposed estimators are also provided using maximal inequalities for martingales. We compare these comprehensive results for general α-divergence with those of Xu and Zhou (2011) who studied empirical Bayes prde in spherically symmetric homoskedastic Gaussian model under KL loss. Our general α-divergence results well reconcile with the KL results in the existing literature.…”

Empirical Bayes Inference for the Parameter of Power Distribution Based on Ranked Set Sampling

“…Empirical Bayes (EB) approach was originally proposed by Robbins [3,4], soon after it had been studied in the literature [5][6][7][8][9]. Up to now, EB methods are commonly based on simple random sampling (SRS).…”