Geometric ergodicity of Pólya-Gamma Gibbs sampler for Bayesian logistic regression with a flat prior

Abstract

The Logistic regression model is the most popular model for analyzing binary data. In the absence of any prior information, an improper flat prior is often used for the regression coefficients in Bayesian logistic regression models. The resulting intractable posterior density can be explored by running Polson et al.'s (2013) data augmentation (DA) algorithm. In this paper, we establish that the Markov chain underlying Polson et al.'s (2013) DA algorithm is geometrically ergodic. Proving this theoretical result is practically important as it ensures the existence of central limit theorems (CLTs) for sample averages under a finite second moment condition. The CLT in turn allows users of the DA algorithm to calculate standard errors for posterior estimates.

Convergence Diagnostics for Markov Chain Monte Carlo

“…Availability of a Markov chain CLT has been demonstrated for myriad MCMC algorithms for common statistical models. Here we provide an incomplete list: linear models Hobert 2012, 2015), generalized linear models including the probit model (Roy andHobert 2007, Chakraborty andKhare 2017), the popular logistic model (Choi andHobert 2013, Wang andRoy 2018c) and the robit model (Roy 2012), generalized linear mixed models including the probit mixed model (Wang and Roy 2018b), and the logistic mixed model (Wang and Roy 2018a), quantile regression models (Khare and Hobert 2012), multivariate regression models (Roy andHobert 2010, Hobert et al 2018), penalized regression and variable selection models (Khare and Hobert 2013, Roy and Chakraborty 2017, Vats 2017. So far we have described the honest MCMC in the context of estimating means of univariate functions.…”

Asynchronous and Distributed Data Augmentation for Massive Data Settings

“…In particular, Choi and Hobert [11] show that the parent PG DA chain is uniformly ergodic (using the marginal β (t) chain) if proper priors are used. Following this work, Wang and Roy [56] show the geometric ergodicity of the PG DA chain with flat priors. The uniform ergodicity proof for the parent DA chain in Choi and Hobert [11] for the binary and proper prior setting is based on a minorization argument on the marginal {β (t) } chain; however, this proof strategy fails for ADDA because the {β (t) } process for Φ ADP G is not Markov.…”

“…The marginal Markov chain {β (t) } of the β draws collected in step PG.2 has the posterior distribution of β in (1) as its invariant distribution [11,56,55].…”

“…As a result, we take a different approach where we establish a geometric drift condition using a function of (β, ω), which is unbounded off compact sets. Wang and Roy [56] follow a similar strategy for the proving geometric ergodicity of the parent DA chain with improper priors, but they focus on the marginal ω chain instead of the (β, ω) chain. Furthermore, the previous two works consider the parent DA chain when the response is binary, whereas Theorem 3.1 deals with the ADDA chain in a more general binomial setting.…”

Approximating the Spectral Gap of the Pólya-Gamma Gibbs Sampler

“…Previous work on the theoretical properties of PG Gibbs samplers includes Choi and Hobert (2013), Choi and Romàn (2017), and Wang and Roy (2018b). Choi and Hobert (2013) analyzed the PG Gibbs sampler under the same normal prior on β and showed that it is uniformly ergodic.…”