Holo-Entropy Based Categorical Data Hierarchical Clustering

Abstract

Clustering high-dimensional data is a challenging task in data mining, and clustering high-dimensional categorical data is even more challenging because it is more difficult to measure the similarity between categorical objects. Most algorithms assume feature independence when computing similarity between data objects, or make use of computationally demanding techniques such as PCA for numerical data. Hierarchical clustering algorithms are often based on similarity measures computed on a common feature space, which is not effective when clustering highdimensional data. Subspace clustering algorithms discover feature subspaces for clusters, but are mostly partition-based; i.e. they do not produce a hierarchical structure of clusters. In this paper, we propose a hierarchical algorithm for clustering high-dimensional categorical data, based on a recently proposed information-theoretical concept named holo-entropy. The algorithm proposes new ways of exploring entropy, holo-entropy and attribute weighting in order to determine the feature subspace of a cluster and to merge clusters even though their feature subspaces differ. The algorithm is tested on UCI datasets, and compared with several state-of-the-art algorithms. Experimental results show that the proposed algorithm yields higher efficiency and accuracy than the competing algorithms and allows higher reproducibility.

Categorical Data Clustering: A Bibliometric Analysis and Taxonomy

“…The detailed analysis and classification are sequentially presented in Section 3.2. Specifically, there are five studies related to hierarchical clustering, with four of them based on rough-set theory (MTMDP [38], MGR [41], MNIG [47], and HPCCD [96]), except the P-ROCK [106]. Two studies focus on agglomerative hierarchical clustering, while the remaining three focus on divisive hierarchical clustering.…”

“…Furthermore, there has been significant advancement in the performance of previous algorithms, shown by the improvement of the min-min-roughness (MMR) algorithm [99]. Divisive, based on an information-theoretic approach MMR, MGR, MDA [103], TR [104] Sun et al ( 2017) HPCCD [96] Agglomerative, based on an information-theoretic approach MGR [41], COOLCAT, LIMBO [105], K-modes,…”

“…(2) Agglomerative Hierarchical Clustering On the other hand, Sun et al [96] developed their algorithm based on agglomerative hierarchical clustering. Their proposed algorithm, named hierarchical projected clustering for categorical data (HPCCD), clusters high-dimensional data using the weighted holoentropy [108] instead of pairwise-similarity-based measures for merging two subclusters.…”

An Entropy-Based Uncertainty Measure for Developing Granular Models

“…In information theory, the measure of uncertainty in a random variable is described by Shannon's entropy in equation in (9). The concept of entropy is always used to represent disorder, chaos, or unpredictability in a dataset [12]. Hence, entropy in the proposed GrC compatibility measure characterizes the reluctance of granules to be merged.…”

An Effective Hybrid Fuzzy Programming Approach for an Entropy-Based Multi-Objective Assembly Line Balancing Problem

“…For this aim the following Shannon entropy (Shannon, 1948) function can be used. This function previously has been used as the objective function of transportation problems (Ojha et al, 2009) and other optimization problems (Sun et al, 2017).…”