K-Means Clustering Versus Validation Measures: A Data-Distribution Perspective

Abstract

K-means is a well-known and widely used partitional clustering method. While there are considerable research efforts to characterize the key features of the K-means clustering algorithm, further investigation is needed to understand how data distributions can have impact on the performance of K-means clustering. To that end, in this paper, we provide a formal and organized study of the effect of skewed data distributions on K-means clustering. Along this line, we first formally illustrate that K-means tends to produce clusters of relatively uniform size, even if input data have varied "true" cluster sizes. In addition, we show that some clustering validation measures, such as the entropy measure, may not capture this uniform effect and provide misleading information on the clustering performance. Viewed in this light, we provide the coefficient of variation (CV) as a necessary criterion to validate the clustering results. Our findings reveal that K-means tends to produce clusters in which the variations of cluster sizes, as measured by CV, are in a range of about 0.3-1.0. Specifically, for data sets with large variation in "true" cluster sizes (e.g., CV > 1.0), K-means reduces variation in resultant cluster sizes to less than 1.0. In contrast, for data sets with small variation in "true" cluster sizes (e.g., CV < 0.3), K-means increases variation in resultant cluster sizes to greater than 0.3. In other words, for the earlier two cases, K-means produces the clustering results which are away from the "true" cluster distributions.

Experiments on Hypothesis "Fuzzy K-Means is Better than K-Means for Clustering"

“…The results proved that the algorithm is reliable. Xiong et al [7] studied K-Means in data -distribution perspective. They described many validation measures such as CV, purity, entropy and F-measure.…”

“…We made an empirical study besides review of literature to prove that the Fuzzy K-Means exhibits better clustering performance than K-Means. The literature on these two and their comparison besides other derivatives of them [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23] and [24] can be found in section IV.…”

Agrupamento e Recomendação de Objetos de Aprendizagem no Padrão IEEE-LOM Considerando Estilos de Aprendizagem

“…Além disso, foram gerados 100 elementos para cada combinação de EAs. Após a geração dos clusters, a qualidade de cada um deles foi avaliada utilizando métricas de validação como Pureza(P), que valida se existem diferentes elementos em cada cluster, e F-Measure (F), que avalia a qualidade de cada cluster com base nos cálculos de precision e recall [Xiong et al 2009]. A Tabela 2 apresenta os resultados obtidos em cada métrica para cada um dos algoritmos testados.…”

A Semi-supervised Learning Framework to Cluster Mixed Data Types

“…Entropy is a commonly used information theoretic external validation measure that measures the purity of the clusters with respect to given external class labels (Xiong et al, 2006). A perfect clustering has an entropy close to 0 which means that every cluster consists of points with only one class label.…”