A Hybrid Scan Gibbs Sampler for Bayesian Models with Latent Variables

Abstract

Gibbs sampling is a widely popular Markov chain Monte Carlo algorithm which is often used to analyze intractable posterior distributions associated with Bayesian hierarchical models. The goal of this article is to introduce an alternative to Gibbs sampling that is particularly well suited for Bayesian models which contain latent or missing data. The basic idea of this hybrid algorithm is to update the latent data from its full conditional distribution at every iteration, and then use a random scan to update the parameters of interest. The hybrid algorithm is often easier to analyze from a theoretical standpoint than the deterministic or random scan Gibbs sampler. We highlight a positive result in this direction from Abrahamsen and Hobert (2018), who proved geometric ergodicity of the hybrid algorithm for a Bayesian version of the general linear mixed model with a continuous shrinkage prior. The convergence rate of the Gibbs sampler for this model remains unknown. In addition, we provide new geometric ergodicity results for the hybrid algorithm and the Gibbs sampler for two classes of Bayesian linear regression models with non-Gaussian errors. In both cases, the conditions under which the hybrid algorithm is geometric are much weaker than the corresponding conditions for the Gibbs sampler. Finally, we show that the hybrid algorithm is amenable to a modified version of the sandwich methodology of Hobert and Marchev (2008), which can be used to speed up the convergence rate of the underlying Markov chain while requiring roughly the same computational effort per iteration.

Asynchronous and Distributed Data Augmentation for Massive Data Settings

“…Other hybrid versions of systematic and random scan Gibbs steps have been considered in the literature. Backlund et al [5] consider a DA setup where the parameter θ is partitioned into two blocks. They construct and study a hybrid sampler that performs a systematic scan step for the entire missing data D at each iteration and updates exactly one of the two parameter blocks with fixed probabilities s and 1 − s, respectively.…”

“…In this setting, from the priors used in (5), only the prior for α given σ 2 is modified to the prior for β given σ 2 as N (0, σ 2 V −1 β ) for some V β 0. We need the following assumptions for our geometric ergodicity result.…”