CUBOS: An Internal Cluster Validity Index for Categorical Data

Abstract

Internal cluster validity index is a powerful tool for evaluating clustering performance. The study on internal cluster validity indices for categorical data has been a challenging task due to the difficulty in measuring distance between categorical attribute values. While some efforts have been made, they ignore the relationship between different categorical attribute values and the detailed distribution information between data objects. To solve these problems, we propose a novel index called Categorical data cluster Utility Based On Silhouette (CUBOS). Specifically, we first make clear the superiority of the paradigm of Silhouette index in exploring the details of clustering results. Then, we raise the Improved Distance metric for Categorical data (IDC) inspired by Category Distance to measure distance between categorical data exactly. Finally, the paradigm of Silhouette index and IDC are combined to construct the CUBOS, which can overcome the aforementioned shortcomings and produce more accurate evaluation results than other baselines, as shown by the experimental results on several UCI datasets.

Understanding the Evaluation Abilities of External Cluster Validity Indices to Internal Ones

“…We first introduce the case dataset used. Then, NC-based method and CAMER are employed to measure the evaluation abilities of seven popular external CVIs [17] to internal CVIs respectively. Finally, we summarize the differences between the two methods, to clarify the superiority of CAMER.…”

Categorical Data Clustering: A Bibliometric Analysis and Taxonomy

“…The publication trend shows an increase, with 2019 emerging as the most productive year. In that year, 11 articles related to clustering were published, covering hierarchical-based [47,48], rough-set-based [49,50], weight-based [51], graph-based [52], a variant of fuzzy clustering [53][54][55], integer linear programming [56], and clustering validity [57]. Moreover, for a deeper comprehension of the citations and publications spanning the period from 2014 to 2023, Table 3 presents the top ten cited articles.…”

“…Table 15 shows the summary of cluster validity. Ng's K-modes [143,220], K-modes [15,221], WKM [176] Bai & Liang (2015) generalized validity function [65] K-modes, CU [222], IE [102] Gao & Wu (2019) IDC, CUBOS [57] CCI [223], CDCS [224], IE, CU, NCC [225] All the validity functions in this study focus on internal validity functions. In 2014, Bai and Liang [39] improved the K-modes algorithm by optimizing its objective function to incorporate both between-cluster separation and within-cluster compactness.…”

“…In 2019, Bai and Liang [57] introduced a study focusing on the generalized validity function. Initially, they examined three existing internal validity functions: K-modes [221], category utility function (CU) [222], and information entropy function (IE) [102].…”

“…Gao and Wu [57] conducted a comprehensive review of existing functions of internal validity indices and, based on that, proposed the categorical data cluster utility based on silhouette (CUBOS). The CUBOS method combines the Silhouette index with an improved distance metric for categorical data (IDC).…”