Geometry-Inference based Clustering-Heuristic: An empirical method for kmeans optimal clusters determination

Abstract

Kmeans is one of the most algorithms that are utilized in data clustering. Number of metrics is coupled with kmeans in order cluster data targeting the enhancement of both locally clusters compactness and the globally clusters separation. Then, before the ultimate data assignment to their corresponding clusters, the selection of the optimal number of clusters should constitute a crucial step in the clustering process. The present work aims to build up a new clustering metric/heuristic that takes into account both space dispersion and inferential characteristics of the data to be clustered. Hence, in this paper, a Geometry-Inference based Clustering (GIC) heuristic is proposed for selecting the optimal numbers of clusters. The conceptual approach proposes the “Initial speed rate” as the main geometric parameter to be inferentially studied. After, the corresponding histograms are fitted by means of classical distributions. A clear linear behaviour regarding the distributions’ parameters was detected according to the number of optimal clusters k* for each of the 14 datasets adopted in this work. Finally, for each dataset, the optimal k* is observed to match with the change-points assigned as the intersection of two clearly salient lines. All fittings are tested using Khi2 tests showing excellent fitting in terms of p-values, and R² also for linear fittings. Then, a change-point algorithm is launched to select k*. To sum up, the GIC heuristic shows a full quantitative aspect, and is fully automated; no qualitative index or graphical techniques are used herein.

Geometry-Inference based Clustering Heuristic: A New kmeans initialization technique for Gaussian data and Experimental Proof of concept

“…Similar studies were performed regarding the simple-effect of the normalization by number of researches such as [27] and [28], and other were dedicated to the simple effect of the clusters' shapes such as Kłopotek et al 2020 which proved kmeans clusters should be hyper(ball)-shaped ones to converge to the global optimum [25], similar statement was proposed by Qiu 2010 [29]. The results of Kłopotek et al 2020 [25] are in total concordance with those of El Khattabi et al 2022 [26] since it was found that normal (Gaussian) standardization are well adapted to Gaussian data-shapes, named in the EL Khattabi's paper as Likely-Gaussian datasets, and in Kłopotek's paper as hyper(ball)-shaped data. Similarly, Hennig 2022 studied nine clustering methods by means of several cluster validation indexes [30], The author measured various individual aspects of the data sets such the scales of data, the clusters separation criterion, and the datasets shapes as mainly the closeness to spatial Gaussian distribution, and so forth.…”

“…In a previous work, the authors of the present paper experimentally proved the importance of data preparation in terms of normalization, and the importance of the data dispersion which was qualified as space data shape, then, these two characteristics were combined with different kmeans metrics for a series of datasets. The findings clearly showed the tri-fold interplay between the latter parameters but also the important sensitivity of these latter on the clustering results [26]. Similar studies were performed regarding the simple-effect of the normalization by number of researches such as [27] and [28], and other were dedicated to the simple effect of the clusters' shapes such as Kłopotek et al 2020 which proved kmeans clusters should be hyper(ball)-shaped ones to converge to the global optimum [25], similar statement was proposed by Qiu 2010 [29].…”

“…A preliminary analysis showed worst quality of results regarding the min-max normalisation. Hence, this is only the standardized normalization of equation ( 2) which was maintained in this work as it was proved by [25] and [26] especially for Gaussian datasets.…”

“…The study of these subsets should reveal the interest of the proposed algorithm and its ability to catch accurate and reliable information for both higher and lower clusters counts. Moreover, it is worth pointing out that datasets should be normalized before starting any operation or computing as it was earlier demonstrated by the authors in a previous publication [26]. Hence, additionally to the row data analysis, two normalization techniques, Equations (1) and (2), were considered in [26].…”

“…Moreover, it is worth pointing out that datasets should be normalized before starting any operation or computing as it was earlier demonstrated by the authors in a previous publication [26]. Hence, additionally to the row data analysis, two normalization techniques, Equations (1) and (2), were considered in [26].…”