Convergence rates of posterior distributions for noniid observations

Abstract

We consider the asymptotic behavior of posterior distributions and Bayes estimators based on observations which are required to be neither independent nor identically distributed. We give general results on the rate of convergence of the posterior measure relative to distances derived from a testing criterion. We then specialize our results to independent, nonidentically distributed observations, Markov processes, stationary Gaussian time series and the white noise model. We apply our general results to several examples of infinite-dimensional statistical models including nonparametric regression with normal errors, binary regression, Poisson regression, an interval censoring model, Whittle estimation of the spectral density of a time series and a nonlinear autoregressive model.Comment: Published at http://dx.doi.org/10.1214/009053606000001172 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Posterior contraction rate for non-parametric Bayesian estimation of the dispersion coefficient of a stochastic differential equation

“…Sections 2 and 3 in Ghosal and van der Vaart (2007). However, we take an alternative approach, that is similar in some respects to the one in Shen and Wasserman (2001) and that relies on results from empirical process theory (see e.g.…”

“…In general we are interested in determination of the 'fastest' rate of decay of ε n , so that (3) still holds. Some general references on derivation of posterior convergence rates under various statistical setups are Ghosal et al (2000), Ghosal and van der Vaart (2007) and Shen and Wasserman (2001). Study of this question parallels the analysis of convergence rates of various estimators in the frequentist literature.…”

Bayes procedures for adaptive inference in inverse problems for the white noise model

“…In the paper [5] it is shown that in this case adaptive Bayesian inference is possible using a hierarchical, conditionally Gaussian prior, while in [35] partially adaptation is shown using Gaussian priors with scale parameter determined by an empirical Bayes method. Other recent papers also exhibit priors that yield rate-adaptive procedures in the direct signal-in-white-noise problem (see for instance [2,13,32,38]), but it is important to note that these papers use general theorems on contraction rates for posterior distributions (as given in [18] for instance) that are not suitable to deal with the truly ill-posed case in which k i → 0 as i → ∞. The reason is that if these general theorems are applied in the inverse case, we only obtain convergence rates relative to the (squared) norm μ → κ 2 i μ 2 i , which is not very interesting.…”

On some aspects of the asymptotic properties of Bayesian approaches in nonparametric and semiparametric models

“…The study of asymptotic properties of Bayesian nonparametric methods methods was initiated by the seminal papers of [Schwartz, 1965,Barron, 1988 then increased significantly after the works of [Barron et al, 1999, Ghosal et al, 2000a. Since then posterior concentration has been extensively studied in various types of models including nonparametric regression [Ghosal and van der Vaart, 2007], Markov models [Tang and Ghosal, 2007], Gaussian time series [Choudhuri et al, 2004. In this paper, we present some recent advances in the study of frequentist properties of Bayesian nonparametric inference.…”