Fast Parallel Expectation Maximization for Gaussian Mixture Models on GPUs Using CUDA

Kumar, Neelam; Satoor, Sanjiv; Buck, Ian

doi:10.1109/hpcc.2009.45

Cited by 65 publications

(33 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several papers [30,31,32,33] have already considered the use of a GPU to calculate these conditional probabilities. This calculation is a weighted sum of Gaussian Probability Density Function (PDF) calculations, and while rearrangement of the algebra is possible the algorithm used is fixed.…”

Section: Computational Loadmentioning

confidence: 99%

See 1 more Smart Citation

Human and computer recognition of regional accents and ethnic groups from British English speech

Hanani

Russell

Carey

2013

Computer Speech & Language

View full text Add to dashboard Cite

Language Recognition Evaluation (LRE) tasks. This system is applied to the problems of regional accent recognition for British English, and ethnic group recognition within a particular accent. We compare the results with human performance and, for accent recognition, the 'text dependent' ACCDIST accent recognition measure. For the fourteen regional accents of British English in the ABI-1 corpus (good quality read speech), our language ID system achieves a recognition accuracy of 86.4%, compared with 95.18% for our best ACCDIST-based system and 58.24% for human listeners. The "Voices across Birmingham" corpus contains significant amounts of telephone conversational speech for the two largest ethnic groups in the city of Birmingham (UK), namely the 'Asian' and 'White' communities. Our language ID system distinguishes between these two groups with an accuracy of 94.3% compared with 90.24% for human listeners. Although direct comparison is difficult, it seems that our language ID system performs much better on the standard twelve class NIST 2003 Language Recognition Evaluation task or the two class ethnic group recognition task than on the fourteen class regional accent recognition task. We conclude that automatic accent recognition is a challenging task for speech technology, and that the use of natural conversational speech may be advantageous for these types of paralinguistic task.One issue with conventional approaches to language ID that use high-order Gaussian Mixture Models (GMMs) and high-dimensional feature vectors is the amount of computing power that they require. Currently, multi-core Graphics Processing Units (GPUs) provide a possible solution at very little cost. In this thesis we also explore the application of GPUs to speech signal and pattern processing, using language ID as a vehicle to demonstrate their benefits. Realisation of the full potential of GPUs requires both effective coding of predetermined algorithms, and, in cases where there is a choice, selection of the algorithm or technique for a specific function that is most able to exploit the properties of the GPU. We demonstrate these principles using the NIST LRE 2003 task, which involves processing over 600 hours of speech. We focus on two parts of the system, namely the acoustic classifier, which is based on a 2048 component GMM, and the acoustic feature extraction process. In the case of the latter we compare a conventional FFT-based analysis with an FIR filter bank, both in terms of their ability to exploit the GPU architecture and language ID performance. With no increase in error rate our GPU based system, with an FIR-based front-end, completes the full NIST LRE 2003 task in 16 hours, compared with 180 hours for the more conventional FFT-based system on a standard CPU (a speed up factor of more than 11).

show abstract

Section: Computational Loadmentioning

confidence: 99%

“…The high computational cost of these processes for large M and substantial amounts of data has already motivated research into the application of GPUs to this task, for example [30,31,32,33].…”

Section: Gmm Probability Computation Optimization Techniquesmentioning

confidence: 99%

Human and computer recognition of regional accents and ethnic groups from British English speech

Hanani

Russell

Carey

2013

Computer Speech & Language

View full text Add to dashboard Cite

show abstract

“…Similarly to the approach of (Machlica et al, 2011) and (Kumar et al, 2009), in our proposal the main loop of the algorithm is implemented sequentially and different CUDA kernels are in charge of running different steps of the algorithm.…”

Section: Rationale Of the Parallelization Approachmentioning

confidence: 99%

“…The work includes a parallel version of the cmeans algorithm. Kumar et al (2009), the authors spread the EM algorithm over six CUDA kernels for a fast parallel parametric estimation of GMM. The work focus is in speeding up the EM algorithm through improvements of the kernels and data organization, not using it for specific problems.…”

Section: Previous Work On Parallel Implementation Of Em and Gmm Learningmentioning

confidence: 99%

Parallel Implementation of Expectation-Maximisation Algorithm for the Training of Gaussian Mixture Models

Araújo¹,

Macedo²,

Chella³

et al. 2014

Journal of Computer Science

View full text Add to dashboard Cite

Most machine learning algorithms need to handle large data sets. This feature often leads to limitations on processing time and memory. The Expectation-Maximization (EM) is one of such algorithms, which is used to train one of the most commonly used parametric statistical models, the Gaussian Mixture Models (GMM). All steps of the algorithm are potentially parallelizable once they iterate over the entire data set. In this study, we propose a parallel implementation of EM for training GMM using CUDA. Experiments are performed with a UCI dataset and results show a speedup of 7 if compared to the sequential version. We have also carried out modifications to the code in order to provide better access to global memory and shared memory usage. We have achieved up to 56.4% of achieved occupancy, regardless the number of Gaussians considered in the set of experiments.

show abstract

“…All c j,i generate a multivariate Gaussian probability distribution using a fast parallel expectation maximization accelerated by GPU [26]:…”

Section: Second Order Featurementioning

confidence: 99%

Time-efficient sparse analysis of histopathological whole slide images

Huang

Veillard

Roux

et al. 2011

Computerized Medical Imaging and Graphics

View full text Add to dashboard Cite

Histopathological examination is a powerful method for the prognosis of critical diseases. But, despite significant advances in high-speed and high-resolution scanning devices or in virtual exploration capabilities, the clinical analysis of Whole Slide Images (WSI) largely remains the work of human experts. We propose an innovative platform in which multi-scale computer vision algorithms perform fast analysis of a histopathological WSI. It relies on specific high and generic low resolution image analysis algorithms embedded in a multi-scale framework to rapidly identify the high power fields of interest used by the pathologist to assess a global grading. GPU technologies as well speed up the global time-efficiency of the system. In a sense, sparse coding and sampling is the keystone of our approach. In terms of validation, we are designing a computer-aided breast biopsy analysis application based on histopathology images and designed in collaboration with a pathology department. The current ground truth slides correspond to about 36,000 high magnification (40X) high power fields. The time processing to achieve automatic WSI analysis is on a par with the pathologist's performance (about ten minutes a WSI), which constitutes by itself a major contribution of the proposed methodology.

show abstract

Fast Parallel Expectation Maximization for Gaussian Mixture Models on GPUs Using CUDA

Cited by 65 publications

References 2 publications

Human and computer recognition of regional accents and ethnic groups from British English speech

Human and computer recognition of regional accents and ethnic groups from British English speech

Parallel Implementation of Expectation-Maximisation Algorithm for the Training of Gaussian Mixture Models

Time-efficient sparse analysis of histopathological whole slide images

Contact Info

Product

Resources

About