Data Integrative Bayesian Inference for Mixtures of Regression Models

Abstract

Summary Modern data collection techniques, which often produce different types of relevant information, call for new statistical learning methods that are adapted to cope with data integration. In the paper Bayesian inference is considered for mixtures of regression models with an unknown number of components, that facilitates data integration and variable selection for high dimensional data. In the approach presented, named data integrative mixture of regressions, data integration is accomplished by introducing a new data allocation scheme that summarizes additional data in the form of an informative prior on latent variables. To cope with high dimensionality, a shrinkage‐type prior is assumed on the regression parameters, and a posteriori variable selection is conducted based on Bayesian credible intervals. Posterior estimation is achieved via a Markov chain Monte Carlo algorithm. The method is validated through simulation studies and illustrated by its performance on real data.

Analysis of Twitter data with the Bayesian fused graphical lasso

“…Multiplying the likelihood ( 12 ) and the prior ( 14 ) we see that the posterior of the latent variables satisfies As the integral in ( 15 ) is not analytically tractable, it can be approximated by Monte Carlo samples as described in [ 16 ].…”

“…Estimation of the mixture model (13) is carried out by first clustering the data points within each group. This is achieved by adopting the Bayesian clustering scheme that assigns informative priors on the component memberships as proposed in [16], and briefly described in Section 2.2.1. Next, the mixture parameters are estimated component-wise.…”

“…To improve the estimation procedure, one may consider making use of available additional information such as cluster information or similarity measurements from other data sources. The proposed data allocation strategy, Data Integrative Chinese Restaurant Process (DI-CRP), is a generalization of the Chinese restaurant process (CRP) with additional flexibility that facilitates incorporating external sample-level information in mixture modeling with an unknown number of components [16]. Like CRP, DI-CRP is a data allocation strategy that explores the conditional distribution of the component membership of one data point given that of the rest of data points.…”

“…As the integral in (15) is not analytically tractable, it can be approximated by Monte Carlo samples as described in [16].…”

“…In particular, we propose a Bayesian Gaussian fused graphical lasso estimation algorithm to estimate group-wise precision matrices that may exhibit network similarities, and augment this with a mixture model to account for heterogeneity of the data within a network. This is done in the spirit of the data integrative Bayesian inference method that we proposed in [16], by forming a new prior distribution on the elements of the precision matrices and obtaining a posterior distribution that resembles the ℓ 1 -penalized likelihood plus a fused penalty. A data allocation scheme is employed to simultaneously uncover the hidden clustering components of the mixture model while estimation of the cluster-specific precision matrices is achieved through column-wise block Gibbs sampling.…”