Understanding and Enhancement of Internal Clustering Validation Indexes for Categorical Data

Abstract

Clustering is one of the main tasks of machine learning. Internal clustering validation indexes (CVIs) are used to measure the quality of several clustered partitions to determine the local optimal clustering results in an unsupervised manner, and can act as the objective function of clustering algorithms. In this paper, we first studied several well-known internal CVIs for categorical data clustering, and proved the ineffectiveness of evaluating the partitions of different numbers of clusters without any inter-cluster separation measures or assumptions; the accurateness of separation, along with its coordination with the intra-cluster compactness measures, can notably affect performance. Then, aiming to enhance the internal clustering validation measurement, we proposed a new internal CVI—clustering utility based on the averaged information gain of isolating each cluster (CUBAGE)—which measures both the compactness and the separation of the partition. The experimental results supported our findings with regard to the existing internal CVIs, and showed that the proposed CUBAGE outperforms other internal CVIs with or without a pre-known number of clusters.

Differences in Assistive Technology Installed for People With Dementia Living at Home Who Have Wandering and Safety Risks

“…To identify groups of people with dementia who have similar AT needs this study employed cluster analysis. This technique is essentially concerned with discovering intrinsic discrete groups within data (27)(28)(29). Reduction of a heterogeneous sample into a number of more homogeneous groups provides a means to organise large quantities of information and facilitates consideration of multiple characteristics (30).…”

Distributed Correlation-Based Clustering Mechanism for Large-Scale Datasets

“…To apply parallel processing to large-scale datasets from distributed computers, it is necessary to consider that each cluster of data processed parallelly cannot have its own uniqueness; it must be able to adapt to current computer resources, and the data volume of each cluster can be adjusted arbitrarily [6,8]. The method adopted here to split the data is to use the initial point filtering method in the K-means++ algorithm for determining the centroid [9].…”