Improving Accuracy of Evolving GMM Under GPGPU-Friendly Block-Evolutionary Pattern

Abstract

As a classical clustering model, Gaussian Mixture Model (GMM) can be the footstone of dominant machine learning methods like transfer learning. Evolving GMM is an approximation to the classical GMM under time-critical or memory-critical application scenarios. Such applications often have constraints on time-to-answer or high data volume, and raise high computation demand. A prominent approach to address the demand is GPGPU-powered computing. However, the existing evolving GMM algorithms are confronted with a dilemma between clustering accuracy and parallelism. Point-wise algorithms achieve high accuracy but exhibit limited parallelism due to point-evolutionary pattern. Block-wise algorithms tend to exhibit higher parallelism. Whereas, it is challenging to achieve high accuracy under a block-evolutionary pattern due to the fact that it is difficult to track evolving process of the mixture model in fine granularity. Consequently, the existing block-wise algorithm suffers from significant accuracy degradation, compared to its batch-mode counterpart: the standard EM algorithm. To cope with this dilemma, we focus on the accuracy issue and develop an improved block-evolutionary GMM algorithm for GPGPU-powered computing systems. Our algorithm leverages evolving history of the model to estimate the latest model order in each incremental clustering step. With this model order as a constraint, we can perform similarity test in an elastic manner. Finally, we analyze the evolving history of both mixture components and the data points, and propose our method to merge similar components. Experiments on real images show that our algorithm significantly improves accuracy of the original general purpose bock-wise algorithm. The accuracy of our algorithm is at least comparable to that of the standard EM algorithm and even outperforms the latter under certain scenarios.

Acoustic Scene Classifier Based on Gaussian Mixture Model in the Concept Drift Situation

“…where, L denotes the model likelihood; 𝑣 denotes the degree of freedom of the model parameters; and 𝑁 denotes the number of training data points. The model with the lowest BIC value is selected because it maximizes the log-likelihood [6]. Algorithm 1 presents the steps of the learning process.…”

“…Moreover, the decision for the frequency of retraining and redeployment is a difficult task. Another promising approach is to use an evolving or incremental learning method [4] [5], where the model is updated when a new subset of data arrives [6]. Each iteration is considered as an incremental step toward revisiting the current model.…”