Gaussian approximation of general non-parametric posterior distributions

Abstract

In a general class of Bayesian nonparametric models, we prove that the posterior distribution can be asymptotically approximated by a Gaussian process. Our results apply to nonparametric exponential family that contains both Gaussian and non-Gaussian regression, and also hold for both efficient (root-n) and inefficient (non root-n) estimation. Our general approximation theorem does not rely on posterior conjugacy, and can be verified in a class of Gaussian process priors that has a smoothing spline interpretation [59,44].In particular, the limiting posterior measure becomes prior-free under a Bayesian version of "under-smoothing" condition. Finally, we apply our approximation theorem to examine the asymptotic frequentist properties of Bayesian procedures such as credible regions and credible intervals.AMS 2000 subject classifications: Primary 62C10 Secondary 62G15, 62G08.

Nonparametric Bayesian Aggregation for Massive Data

“…In this section, we present a general nonparametric Bayesian framework as developed in [24]. Specifically, we consider a class of nonparametric exponential family that covers both Gaussian and non-Gaussian regression, and further assign a class of Gaussian process priors to this family.…”

“…As shown in Section 2 of [24], Assumption A1 holds for Gaussian regression, Binary regression, Binomial regression and Poisson regression models with different values of C 0 , C 1 and C 2 . Furthermore, Proposition A.1 in Appendix guarantees that there exists an underlying eigen-system (ϕ ν (•), ρ ν ) (defined in (A.1)) that simultaneously diagonalizes two bilinear forms V and U , where V (g, g) := E{ Ä(f 0 (X))g(X)g(X)} and U (g, g) := 1 0 g (m) (x)g (m) (x)dx for any g, g ∈ S m (I).…”

“…Furthermore, Proposition A.1 in Appendix guarantees that there exists an underlying eigen-system (ϕ ν (•), ρ ν ) (defined in (A.1)) that simultaneously diagonalizes two bilinear forms V and U , where V (g, g) := E{ Ä(f 0 (X))g(X)g(X)} and U (g, g) := 1 0 g (m) (x)g (m) (x)dx for any g, g ∈ S m (I). We next present a class of Gaussian process priors introduced in [24]:…”

“…Theorem 3.5 in [24] shows that P 0j (conditional on D j ) is induced by a Gaussian process, denoted as W j , in the sense that P 0j (S) = P (W j ∈ S|D j ) for any S ∈ S. Define…”

“…A fundamental theory underlying Bayesian aggregation is a uniform version of nonparametric Gaussian approximation theorem, also called as Bernstein-von Mises theorem. Developed based on our recent work [24], this theory states that a sequence of individual posterior distributions converge to Gaussian processes uniformly over the number of subsets.…”