A comparison of clustering quality indices using outliers and noise

Abstract

Quality indices in clustering are used not only to assess the quality of the partitions but also to determine the number of clusters in the final result. When these indices are evaluated in a case study, real data conditions or different clustering algorithms are seldom taken into account. Here, some of the standard indices used in the literature are compared using more realistic databases that include outliers or noisy dimensions, which is more like a real problem-solving approach. Besides, three different clustering methods are used in an attempt to identify different behaviours. Also, the performance of the quality index-clustering algorithm tandem is compared to random grouping, with the aim of running an additional check. The indices are ranked, and index-based conclusions are drawn for all the scenarios.

Fuzziness parameter selection in fuzzy c-means: The perspective of cluster validation

“…However, the process of cluster validation is a complex process [35]. Though various CVIs have been proposed [27,28,[36][37][38], no CVI performs better than all the others. Similarly, not one CVI is suitable and can perform effectively on all kinds of data sets [38][39][40][41][42][43][44][45].…”

“…The number of clusters is a basic and necessary input parameter of FCM algorithm, which is generally obtained by various CVIs or prior information from domain knowledge. For a data set without any prior information, the first step to use our method is to determine the optimal number of clusters using the selected most effective CVI from various CVIs [27,28,[36][37][38]. We also note that other factors such as the basic principles of FCM algorithm, the performance of CVIs, and the characteristics of data sets will also affect the clustering process, and thus affect the selection of m. In our future work, we will focus on these factors.…”

Pattern Recognition and Anomaly Detection by Self-Organizing Maps in a Multi Month E-nose Survey at an Industrial Site

“…Supervised and semisupervised classification algorithms can be implemented if respectively all or part of the data are associated to class labels; in these cases a fusion of multiple classifier can lead to improved classification performance [41]. The issue of identification of an appropriate number of clusters, among the number of possible partitionings, is addressed computing validity indexes [42] as the Davies-Bouldin index [43], which compares S c within-cluster distance and d ce between clusters distance. Other effective validity indexes are the Calinski-Harabasz index and the Gamma index [44].…”

Method evaluation, parameterization, and result validation in unsupervised data mining: A critical survey

“…Guerra, Robles, Bielza, and Larrañaga () also added noise or outliers to artificial data. They evaluated five internal criteria, Silhouette, CH, C‐Index, DB, Gamma, using artificial data with different yet relatively low dimensionality, number of clusters, outliers, and noise.…”