Parallel Markov chain Monte Carlo for non‐Gaussian posterior distributions

Abstract

Recent developments in big data and analytics research have produced an abundance of large data sets that are too big to be analyzed in their entirety, due to limits on computer memory or storage capacity. To address these issues, communication-free parallel Since our method estimates only marginal densities, there is no limitation on the number of model parameters analyzed. Our procedure is suitable for Bayesian models with unknown parameters with fixed dimension in continuous parameter spaces.3

Parallel Markov chain Monte Carlo for Bayesian dynamic item response models in educational testing

“…By assuming that the subposterior densities are independent, the subposterior samples are combined to estimate the full data posterior density by the following (Scott et al., ; Neiswanger et al., ; Wang & Dunson, ; Miroshnikov et al., ): …”

“…There are many recent techniques for combining subset MCMC samples, including sample averaging, weighted averaging (consensus Monte Carlo; Scott et al., ), kernel smoothing (Neiswanger et al ), and Weierstrass sampling (Wang and Dunson, ). These methods work well when the posterior distribution is Gaussian; however, these methods do not perform as well when the posterior is non‐Gaussian (Miroshnikov et al ). Recently, a direct density product method was introduced for non‐Gaussian posteriors (Miroshnikov et al ).…”

“…These methods work well when the posterior distribution is Gaussian; however, these methods do not perform as well when the posterior is non‐Gaussian (Miroshnikov et al ). Recently, a direct density product method was introduced for non‐Gaussian posteriors (Miroshnikov et al ). This method works as follows.…”

“…These subset posterior densities are then multiplied together across a set of grid values that spans the range of the parameter values; this creates an unnormalized marginal density, which is then normalized through polynomial interpolation. The resulting normalized combined marginal posterior density approximates the full data marginal posterior density for each parameter; see Miroshnikov et al () for full details of this method.…”

Asymptotic properties of parallel Bayesian kernel density estimators

“…To address these issues, numerous alternative communication-free parallel MCMC methods have been developed for Bayesian analysis of big data. These methods partition data into subsets, perform independent Bayesian MCMC analysis on each subset, and combine the subset posterior samples to estimate the full data posterior; see [23,19,17].…”