Honest Exploration of Intractable Probability Distributions via Markov Chain Monte Carlo

Abstract

Two important questions that must be answered whenever a Markov chain Monte Carlo (MCMC) algorithm is used are (Q1) What is an appropriate burn-in? and (Q2) How long should the sampling continue after burn-in? Developing rigorous answers to these questions presently requires a detailed study of the convergence properties of the underlying Markov chain. Consequently, in most practical applications of MCMC, exact answers to (Q1) and (Q2) are not sought. The goal of this paper is to demystify the analysis that leads to honest answers to (Q1) and (Q2). The authors hope that this article will serve as a bridge between those developing Markov chain theory and practitioners using MCMC to solve practical problems.The ability to address (Q1) and (Q2) formally comes from establishing a drift condition and an associated minorization condition, which together imply that the underlying Markov chain is geometrically ergodic. In this article, we explain exactly what drift and minorization are as well as how and why these conditions can be used to form rigorous answers to (Q1) and (Q2). The basic ideas are as follows. The results of Rosenthal (1995) and Roberts and Tweedie (1999) allow one to use drift and minorization conditions to construct a formula giving an analytic upper bound on the distance to stationarity. A rigorous answer to (Q1) can be calculated using this formula. The desired characteristics of the target distribution are typically estimated using ergodic averages. Geometric ergodicity of the underlying Markov chain implies that there are central limit theorems available for ergodic averages (Chan and Geyer 1994). The regenerative simulation technique (Mykland, Tierney and Yu, 1995;Robert, 1995) can be used to get a consistent estimate of the variance of the asymptotic normal distribution. Hence, an asymptotic standard error can be calculated, which provides an answer to (Q2) in the sense that an appropriate time to stop sampling can be determined. The methods are illustrated using a Gibbs sampler for a Bayesian version of the one-way random effects model and a data set concerning styrene exposure.

Micro-local analysis for the Metropolis algorithm

“…Observe that, without some information about a(x) and γ, 2.8 says little more than 2.6. More refined estimates have been developed for special cases ( [JH01]), . These have come to be called "honest bounds" because specific a(x) > 0 and γ are given.…”

An Auto-validating Rejection Sampler for Differentiable Arithmetical Expressions: Posterior Sampling of Phylogenetic Quartets

“…The standard approaches to sampling from the posterior density, especially over phylogenetic trees, rely on Markov chain Monte Carlo (MCMC) methods. Despite their asymptotic validity, it is nontrivial to guarantee that an MCMC algorithm has converged to stationarity [1], and thus MCMC convergence diagnostics on phylogenetic tree spaces are heuristic and may lead to meaningless estimates [2].…”

“…Informally, MRS [5,6] partitions the domain into boxes and uses interval analysis to rigorously enclose the range of the target shape in each box; then it uses as envelope the piecewise constant function given by the upper bound of the range in each box. More formally, the method employs the natural interval extension of the target posterior shape f (t) : T → R to produce rigorous enclosures of the range of f over each interval vector or box in an adaptive partition T := {t (1) , t (2) , . .…”

Using a Markov Chain to Construct a Tractable Approximation of an Intractable Probability Distribution

“…To be specific, minorization and drift conditions must be established for the Markov chain. See Jones and Hobert (2001) for a simple introduction to these concepts. Hobert and Robert (2004) presented an alternative solution to the problem described above, which, unfortunately, also requires minorization and drift conditions.…”

“…Indeed, these authors assume that s(x) has the specific form εI C (x). There are practical advantages to working with the more general form of minorization (Jones and Hobert, 2001).…”