Internal evaluation criteria for categorical data in hierarchical clustering

Abstract

The paper compares 11 internal evaluation criteria for hierarchical clustering of categorical data regarding a correct number of clusters determination. The criteria are divided into three groups based on a way of treating the cluster quality. The variability-based criteria use the within-cluster variability, the likelihood-based criteria maximize the likelihood function, and the distance-based criteria use distances within and between clusters. The aim is to determine which evaluation criteria perform well and under what conditions. Different analysis settings, such as the used method of hierarchical clustering, and various dataset properties, such as the number of variables or the minimal between-cluster distances, are examined. The experiment is conducted on 810 generated datasets, where the evaluation criteria are assessed regarding the optimal number of clusters determination and mean absolute errors. The results indicate that the likelihood-based BIC1 and variability-based BK criteria perform relatively well in determining the optimal number of clusters and that some criteria, usually the distance-based ones, should be avoided.

The positive–negative–competence (PNC) model of psychological responses to representations of robots

“…However, we included many items that were overlapping or similar (for example, 'performs actions' and 'performs certain actions') to ensure that the potential content space of robot characteristics was sampled in detail (for the final list of 277 characteristics see Supplementary Table 3). The characteristics, as sorted into categories by Sample 2 participants, were subjected to hierarchical cluster analysis for categorical data [114][115][116] : a dissimilarity matrix was computed using Gower's distance 198,199 , clusters were produced using Ward's linkage method 200,201 and the optimal number of clusters was determined via the mean silhouette width approach using the partitioning around medoids algorithm 114,202,203 . The five clusters that emerged were then arranged into the robot definition (Table 2).…”