Bayesian Inference for Logistic Models Using Pólya–Gamma Latent Variables

Abstract

We propose a new data-augmentation strategy for fully Bayesian inference in models with binomial likelihoods. The approach appeals to a new class of Pólya-Gamma distributions, which are constructed in detail. A variety of examples are presented to show the versatility of the method, including logistic regression, negative binomial regression, nonlinear mixed-effects models, and spatial models for count data. In each case, our data-augmentation strategy leads to simple, effective methods for posterior inference that: (1) circumvent the need for analytic approximations, numerical integration, or Metropolis-Hastings; and (2) outperform other known data-augmentation strategies, both in ease of use and in computational efficiency. All methods, including an efficient sampler for the Pólya-Gamma distribution, are implemented in the R package BayesLogit.In the technical supplement appended to the end of the paper, we provide further details regarding the generation of Pólya-Gamma random variables; the empirical benchmarks reported in the main manuscript; and the extension of the basic dataaugmentation framework to contingency tables and multinomial outcomes.

Multiscale Spatial Density Smoothing: An Application to Large-Scale Radiological Survey and Anomaly Detection

“…With this simple data-augmentation trick from (Polson et al, 2013), the above model can be fit by Gibbs sampling, with the following conditionals. We let − denote all other variables being conditioned upon, and κ (s) = y (s) − m (s) /2.…”

“…We use the gammalasso because it also has a simple interpretation as a mixture of Laplace priors, making the connection with the graph fused lasso explicit, and because our experiments with other choices led to essentially equivalent performance. We represent the binomial likelihood using the Pólya-Gamma data augmentation scheme described in Polson et al (2013). This yields conditionally conjugate MCMC updates for all model parameters.…”

Complex Latent Variable Modeling in Educational Assessment

“…The logistic counterpart of the normal ogive model known as the Rasch model has recently been addressed by Polson et al (2013), who propose a Gibbs sampling procedure for logistic models analog to the data augmentation procedure. The latent responses follow a Polya-Gamma distribution, and if the prior distributions p(θ ) and p(b) are normal distributions, it follows that the posterior distributions of the person and item parameters can again be derived using normal linear model results.…”

Discriminative Bayesian Nonparametric Clustering

“…Separated clusters often result in distinct and more interpretable clusters and the number of clusters are smaller. Previous work has argued that discriminative clustering is desirable in various tasks, e.g., subspace selection analysis [De la Torre and Kanade, 2006;Ye et al, 2008], computer vision [Joulin et al, 2010], unsupervised regression [Krause et al, 2010] and factor modeling [Henao et al, 2014].…”

“…As the number of clusters is often unknown and growing over time, modern Bayesian nonparametric (BNP) clustering methods have the advantages of automatically identifying the suitable number of clusters -most popular models include the Dirichlet process mixture models (DPM) [Ferguson, 1973;Antoniak, 1974;Sethuraman, 1994;Ghahramani, 2012]. While DPM and models alike have been successful, they are strictly generative and the clustering outcomes favor the similarity of data points within a cluster, but do not explicitly model the discrimination among clusters.…”