Accelerating Collapsed Variational Bayesian Inference for Latent Dirichlet Allocation with Nvidia CUDA Compatible Devices

Masada, Tomonari; Hamada, Takashi; Shibata, Yuichiro; Oguri, Kazuta

doi:10.1007/978-3-642-02568-6_50

Cited by 11 publications

(12 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Masada, et al [8] and Yan, et al [13] made use of GPGPU for the parallel inference of LDA. Their focus was on the limit of memory size of GPGPU, which is dierent from our motivation of this paper.…”

Section: Fast Inference Methods For Ldamentioning

confidence: 99%

MPI/OpenMP hybrid parallel inference for Latent Dirichlet Allocation

Tora

Eguchi

2011

Proceedings of the Third Workshop on Large Scale Data Mining: Theory and Applications

View full text Add to dashboard Cite

In recent years, probabilistic topic models have been applied to various kinds of data including text data, and its eectiveness has been demonstrated. Latent Dirichlet Allocation (LDA) is one of the well-known topic models. Variational Bayesian inference or collapsed Gibbs sampling is often employed to estimate parameters in LDA; however, these inference methods require high computational cost for large-scale data. Therefore, high eciency technology is needed for this purpose. In this paper, we make use of parallel computation technology for the sake of ecient collapsed Gibbs sampling inference for LDA. We assume to use a shared memory cluster (SMP cluster), which is widely used in recent years. In prior work of parallel inference for LDA, either MPI or OpenMP has been used alone. On the other hand, for a SMP cluster it is more suitable to adopt hybrid parallelization that uses message passing for communication between SMP nodes and loop directives for parallelization within each SMP node. In this paper, we developed a MPI/OpenMP hybrid parallel inference method for LDA, and achieved remarkable speedup under various settings of a SMP cluster.

show abstract

Section: Fast Inference Methods For Ldamentioning

confidence: 99%

MPI/OpenMP hybrid parallel inference for Latent Dirichlet Allocation

Tora

Eguchi

2011

Proceedings of the Third Workshop on Large Scale Data Mining: Theory and Applications

View full text Add to dashboard Cite

show abstract

“…Note that there are NaN data in FDR of VLR, which can not be shown in Figure 2 and Figure 3. This is due to the fact that, when the sample size is too small compared with the number of regression coefficients, VLR will infers all coefficients to some small values, thus all the regressors will be determined 10 as 0, therefore VLR will have '0 n  , and the FDR will be NaN.…”

Section: Part Of Prior Mean Of the Nonzero Coefficients In Models Arementioning

confidence: 99%

“…Meanwhile, variational approximation techniques have also been extensively studied to solve the linear regression models [8][9][10][11][12]. The variational approximation methods assume specific priori distributions, adopt the approximation to the joint posterior distribution, and provide an analytical iterative expression for the posterior probability of the unknown regressors.…”

Section: Introductionmentioning

confidence: 99%

Variational Inference of Linear Regression with Nonzero Prior Means

Dong

Wang

2014

Communications in Statistics - Simulation and Computation

View full text Add to dashboard Cite

In this paper, we employ the variational Bayesian method to study the parameter estimation problems of linear regression model, wherein some regressors are of Gaussian distribution with non-zero prior means. We obtain an analytical expression of the posterior parameter distribution, and then propose an iterative algorithm for the model. Simulations are carried out to test the performance of the proposed algorithm, and the simulation results confirm both the effectiveness and the reliability of the proposed algorithm.

show abstract

“…While the above formula is computationally intractable in most cases, approximations exist. Using collapsed Gibbs sampling with randomization, 26x speedup has been achieved over a serial implementation [57], and 196x speedup by using collapsed variational Bayesian [31,57], which is a deterministic method that avoids randomization.…”

Section: Text Mining On Gpusmentioning

confidence: 99%

Accelerating text mining workloads in a MapReduce-based distributed GPU environment

Wittek

Daranyi

2013

Journal of Parallel and Distributed Computing

View full text Add to dashboard Cite

Scientific computations have been using GPU-enabled computers successfully, often relying on distributed nodes to overcome the limitations of device memory. Only a handful of text mining applications benefit from such infrastructure. Since the initial steps of text mining are typically data-intensive, and the ease of deployment of algorithms is an important factor in developing advanced applications, we introduce a flexible, distributed, MapReducebased text mining workflow that performs I/O-bound operations on CPUs with industry-standard tools and then runs compute-bound operations on GPUs which are optimized to ensure coalesced memory access and effective use of shared memory. We have performed extensive tests of our algorithms on a cluster of eight nodes with two NVidia Tesla M2050 attached to each, and we achieve considerable speedups for random projection and self-organizing maps.

show abstract

Accelerating Collapsed Variational Bayesian Inference for Latent Dirichlet Allocation with Nvidia CUDA Compatible Devices

Cited by 11 publications

References 12 publications

MPI/OpenMP hybrid parallel inference for Latent Dirichlet Allocation

MPI/OpenMP hybrid parallel inference for Latent Dirichlet Allocation

Variational Inference of Linear Regression with Nonzero Prior Means

Accelerating text mining workloads in a MapReduce-based distributed GPU environment

Contact Info

Product

Resources

About