Convergence Rates and Asymptotic Standard Errors for Markov Chain Monte Carlo Algorithms for Bayesian Probit Regression

Abstract

Consider a probit regression problem in which "Y" 1, …, "Y" "n" are independent Bernoulli random variables such that where "x" "i" is a "p"-dimensional vector of known covariates that are associated with "Y" "i" , "&bgr;" is a "p"-dimensional vector of unknown regression coefficients and Φ(·) denotes the standard normal distribution function. We study Markov chain Monte Carlo algorithms for exploring the intractable posterior density that results when the probit regression likelihood is combined with a flat prior on "&bgr;". We prove that Albert and Chib's data augmentation algorithm and Liu and Wu's PX-DA algorithm both converge at a geometric rate, which ensures the existence of central limit theorems for ergodic averages under a second-moment condition. Although these two algorithms are essentially equivalent in terms of computational complexity, results of Hobert and Marchev imply that the PX-DA algorithm is theoretically more efficient in the sense that the asymptotic variance in the central limit theorem under the PX-DA algorithm is no larger than that under Albert and Chib's algorithm. We also construct minorization conditions that allow us to exploit regenerative simulation techniques for the consistent estimation of asymptotic variances. As an illustration, we apply our results to van Dyk and Meng's lupus data. This example demonstrates that huge gains in efficiency are possible by using the PX-DA algorithm instead of Albert and Chib's algorithm. Copyright 2007 Royal Statistical Society.

Gibbs sampling, conjugate priors and coupling

“…Iterating this gives a Markov chain x, x , x , · · · with f as stationary density under mild conditions [2,36]. The running time analysis of the Gibbs sampler (how many steps should the chain be run to be close to stationarity) is an active area of research [13,19,20,22,24,29,30,31,32]. This paper considers two component examples of the form…”

Estimation of Bayes Factors in a Class of Hierarchical Random Effects Models Using a Geometrically Ergodic MCMC Algorithm

“…The exceptions include the MCMC algorithms studied by Mengersen and Tweedie (1996), Roberts and Tweedie (1996), Hobert and Geyer (1998), Roy and Hobert (2007), Marchev and Hobert (2004), Roberts and Rosenthal (1999), Jarner and Hansen (2000), and Papaspiliopoulos and Roberts (2008). (It should be noted that there is not a single result in the literature that gives geometric ergodicity for any of the Markov chains used to estimate the posterior distribution in mixture of Dirichlet process models such as the one we considered in Section 4.…”

A novel sandwich algorithm for empirical Bayes analysis of rank data

“…However, it may be possible to find an r that leads to a huge improvement in mixing despite the fact that the computational cost of drawing from r is negligible relative to the cost of drawing from p(θ|π, y) and p(π|θ, y) (see e.g. Roy , 2014;Roy and Hobert , 2007;Hobert, Roy and Robert , 2011). We propose the following SA improving the θ subchain, {θ (m) } m≥0 , of the Gibbs sampler.…”

“…g .) The reducibility of r is common among efficient sandwich chains (Roy , 2012b;Hobert, Roy and Robert , 2011;Roy and Hobert , 2007).…”