High-dimensional hierarchical models and massively parallel computing

Abstract

Markov chain Monte Carlo (MCMC) is the predominant tool used in Bayesian parameter estimation for hierarchical models. When the model expands due to an increasing number of hierarchical levels, number of groups at a particular level, or number of observations in each group, a fully Bayesian analysis via MCMC can easily become computationally demanding, even intractable. We illustrate how the steps in an MCMC for hierarchical models are predominantly one of two types: conditionally independent draws or low-dimensional draws based on summary statistics of parameters at higher levels of the hierarchy. Parallel computing can increase efficiency by performing embarrassingly parallel computations for conditionally independent draws and calculating the summary statistics using parallel reductions. During the MCMC algorithm, we record running means and means of squared parameter values to allow convergence diagnosis and posterior inference while avoiding the costly memory transfer bottleneck. We demonstrate the effectiveness of the algorithm on a model motivated by next generation sequencing data, and we release our implementation in R packages fbseq and fbseqCUDA.

Applications of Bayesian hierarchical models in gene expression and product reliability

“…. 55 Table 3.3: Two-way tables comparing the posterior probabilities of low-parent gene heterosis for all genes in the Paschold et al (2012) data set found by BNP and Landau and Niemi (2016). .…”

“…. 58 Table 3.4: Two-way tables comparing the posterior probabilities of high-parent heterosis for all genes in the Paschold et al (2012) data set found by BNP and Landau and Niemi (2016 Figure 2.1: Bivariate histograms of independent estimates of effects and standard deviation obtained by ordinary least squares for 36,081 genes (data from Paschold et al (2012)). This example shows that random effects models assuming normality and/or independence may not be suitable for modeling gene expression hierarchically.…”

“…Because of the dimensionality of the problem is large, it is cumbersome to save all of the MCMC samples. Following the approach in Landau and Niemi (2016), we preselect a small number of parameters for which we do save each iteration, but for the rest we first determine which functions of those parameters are of interest and use online algorithms that use running sums to update our estimates of those functions. The class of estimates we calculate fall into two categories, 1. expectations of the component-specific parameters and their squares, i.e.…”

“…Some popular methods use hierarchical models but make unrealistic assumptions, which in some cases could lead to inefficient partial pooling of information. We propose the application of a Bayesian nonparametric approach (Ishwaran and James, 2001;Liu et al, 2015) that avoids these assumptions and implement a Gibbs sampler on a GPU which allows posterior inference to be computationally feasible (Suchard et al, 2010;Landau and Niemi, 2016).…”

“…This suggests that between-gene information is available not only to regulate genelevel assessment of variance, but also the estimated effects themselves. While others used random effects models for individual regression components (Love et al, 2014;Landau and Niemi, 2016)], they depend on assumptions about the underlying distribution of effects, such as independence, which are contradicted by the data.…”