Online Clustering of High-Dimensional Trajectories under Concept Drift

Abstract

Abstract. Historical transaction data are collected in many applications, e.g., patient histories recorded by physicians and customer transactions collected by companies. An important question is the learning of models upon the primary objects (patients, customers) rather than the transactions, especially when these models are subjected to drift.We address this problem by combining advances of online clustering on multivariate data with the trajectory mining paradigm. We model the measurements of each individual primary object (e.g. its transactions), taken at irregular time intervals, as a trajectory in a high-dimensional feature space. Then, we cluster individuals with similar trajectories to identify sub-populations that evolve similarly, e.g. groups of customers that evolve similarly or groups of employees that have similar careers.We assume that the multivariate trajectories are generated by drifting Gaussian Mixture Models. We study (i) an EM-based approach that clusters these trajectories incrementally as a reference method that has access to all the data for learning, and propose (ii) an online algorithm based on a Kalman filter that efficiently tracks the trajectories of Gaussian clusters. We show that while both methods approximate the reference well, the algorithm based on a Kalman filter is faster by one order of magnitude compared to the EM-based approach.

Detecting changes of clustering structures using normalized maximum likelihood coding

“…(8). If the RNML code-length is replaced with BIC, the sequential DMS criterion is given as follows:…”

“…[8] proposed a method of tracking clutering changes using the EM algorithm and Kalman filters. Our work is different from Krempl's one in that the former is concerned with changes of the number of clusters while the latter is concerned with parameter trajectories keeping the number of clusters fixed.…”

“…Next, we show how to compute the RNML code-length (4) for a GMM as in the sequential DMS criterion (8).…”

Data Mining Models of High Dimensional Data Streams, and Contemporary Concept Drift Detection Methods: a Comprehensive Review

“…The work [47] presents similar effects of construct KDQ trees. The process of tracking online clustering trajectories contributed by [48]. The use of cluster deviation leads to detect the data stream evolution proposed by [49].…”

“…[42], [43], [44], [45] Nonparametric methods Not depends on data distribution assumptions [46], [47] Multivariate Estimating the kernel density to identify the drift in concept. [48] Multivariate Tracking online clustering trajectories; [49] Multivariate Cluster deviation [26] Multivariate k-clustering with input data, using cluster populations to estimate the data distribution. [50] Multivariate t-squared test, computes the probability of distribution similarity between the pair of time-windows in sequence.…”