Incremental Cluster Validity Indices for Online Learning of Hard Partitions: Extensions and Comparative Study

Abstract

Validation is one of the most important aspects of clustering, but most approaches have been batch methods. Recently, interest has grown in providing incremental alternatives. This paper extends the incremental cluster validity index (iCVI) family to include incremental versions of Calinski-Harabasz (iCH), I index and Pakhira-Bandyopadhyay-Maulik (iI and iPBM), Silhouette (iSIL), Negentropy Increment (iNI), Representative Cross Information Potential (irCIP) and Representative Cross Entropy (irH), and Conn Index (iConn Index). Additionally, the effect of under-and over-partitioning on the behavior of these six iCVIs, the Partition Separation (PS) index, as well as two other recently developed iCVIs (incremental Xie-Beni (iXB) and incremental Davies-Bouldin (iDB)) was examined through a comparative study. Experimental results using fuzzy adaptive resonance theory (ART)-based clustering methods showed that while evidence of most under-partitioning cases could be inferred from the behaviors of all these iCVIs, over-partitioning was found to be a more challenging scenario indicated only by the iConn Index. The expansion of incremental validity indices provides significant novel opportunities for assessing and interpreting the results of unsupervised learning. PREPRINT SUBMITTED TO ARXIV.ORG 2 I. INTRODUCTION Cluster validation [1] is a critical topic in cluster analysis. It is crucial to assess the quality of the partitions detected by clustering algorithms when there is no class label information. Different clustering solutions may be found by distinct algorithms, or even by the same algorithm subjected to different hyper-parameters or a different input presentation order [2], [3]. Cluster validity indices (CVIs) perform the role of evaluators of such solutions. CVIs typically exhibit a trade-off between measures of compactness (within-cluster scatter) and isolation (between-cluster separation) [2]. Numerous examples of such criteria have been presented in the literature; for comprehensive reviews and experimental studies the interested reader can go to [4]-[11]. Recently, incremental cluster validity indices (iCVIs) have been developed to track the effectiveness of online clustering methods over data streams [12]-[15]. To enable cluster validation in such applications, a recursive formulation of compactness was introduced in [12], [13]. This strategy has been used to develop incremental versions of four CVIs so far [15]: viz., incremental Davies-Bouldin (iDB) [12], [13], incremental Xie-Beni (iXB) [12], [13] and modified Dunn's indices [16]. Particularly, the behavior of iXB and iDB are analyzed in both accurately and poorly partitioned data sets in [12], [13], whereas the studies in [14], [15] only investigate the iDB's behavior in cases where online clustering algorithms accurately detect data structures, i.e., when they yield high performing experimental results. Therefore, the contributions of this work are three-fold: (1) presenting incremental versions of six additional CVIs (thereby extending the family of iC...

Meta-iCVIs: Ensemble Validity Metrics for Concise Labeling of Correct, Under- or Over-Partitioning in Streaming Clustering

“…[9] and [3] introduced and evaluated the incremental Davies-Bouldin (iDB) and Xie-Beni (iXB) indices for their applicability to accurately and poorly partitioned data sets while [10] and [11] investigated the behavior of iDB in cases in which the clustering methods accurately detected structures present in the data. [4] extended this family of iCVIs to include six additional versions. In that work, it was found that certain iCVIs are more effective at identifying certain partition conditions.…”

“…Indeed, methods have been proposed which focus solely on the proper selection of an CVI for optimal results on a particular data set [13]. However, an extension of [4] found that a single iCVI can serve as a vigilance mechanism for Fuzzy ART [14] as well as for TopoARTMAP [15]. The former study investigated iCH, iWB, iXB, iDB, incremental Pakhira-Bandyopadhyay-Maulik (iPBM), and incremental Negentropy Increment (iNI).…”

“…In all previous experiments, data samples were presented in a sorted ordering based on their cluster labels. The rationale behind this is to allow comparisons with published works such as [4], which only evaluated performance of iCVIs for the condition Results are presented as a confusion matrix where an ideal result would be a diagonal matrix.…”

ClusterValidityIndices.jl: Batch and Incremental Metrics for Unsupervised Learning

“…CVIs are useful as one of the only methods of determining the performance of a clustering algorithm in the absence of explicit labels (Arbelaitz et al, 2013). Furthermore, ICVIs can measure the performance of clustering algorithms as they are running in a computationally tractable manner, which is incredibly useful in a variety of streaming clustering applications (Brito Da Silva et al, 2020).…”

“…In fact, it is often the trendlines of these values that provide the most information about the clustering process rather than the values themselves. incremental variants that are proven to be mathematically equivalent to their batch counterparts (Brito Da Silva et al, 2020). These incremental CVIs (ICVIs) mitigate the computational overhead of computing these metrics online, such as in a streaming clustering scenarios.…”