Clustering large data sets described with discrete distributions and its application on TIMSS data set

Abstract

Symbolic data analysis is based on special descriptions of data-symbolic objects. Such descriptions preserve more detailed information about the data than the standard representations with mean values. A special kind of symbolic object is also representation with distributions. In the clustering process this representation enables us to consider the variables of all types at the same time.We present two clustering methods based on the data descriptions with discrete distributions: the adapted leaders method and the adapted agglomerative hierarchical clustering Ward's method. Both methods are compatible-they can be viewed as two approaches for solving the same clustering optimization problem. In the obtained clustering, the leader is assigned to each cluster. The descriptions of the leaders offer simple interpretation of the clusters' characteristics. The leaders method enables us to efficiently solve clustering problems with a large number of units; while the agglomerative method is applied on the obtained leaders and enables us to decide upon the right number of clusters on the basis of the corresponding dendrogram.Both methods were successfully applied in analyses of different data sets. In this paper, an application on the Trends in International Mathematics and Science Study (TIMSS) data set is presented. The descriptions with distributions enable us to combine two data sets: answers of teachers and answers of their students, into one data set. The descriptions of the obtained clusters enable us to interpret the results in a more understandable way. 

Classifying scientific disciplines in Slovenia: A study of the evolution of collaboration structures

“…Recent developments of multivariate techniques dealing with symbolic data include Billard and Diday (). A clustering procedure to deal precisely with our type of symbolic data has been developed (Korenjak‐Černe, Kejžar, & Batagelj, ; Korenjak‐Černe, Batagelj, & Japelj Pavešić, ) and has been termed a modal multivalued symbolic data clustering procedure. Several clustering procedures for symbolic data have been adapted and implemented in R (R Development Core Team, ) within the package Clamix (Batagelj & Kejžar, ).…”

A weighted clustering of population pyramids for the world's countries, 1996, 2001, 2006

“…The measure of closeness is the dissimilarity between each pair of clusters, which must be determined between the new (merged) cluster and the remaining clusters after each merge. We used the weighted clustering method based on the method presented in Batagelj et al (2011) for the case of age-sex distributions.…”

“…In this paper, we present the results of a weighted clustering method based on richer descriptions than classical data descriptions (Batagelj et al 2011). The main advantage of this method is that the clusters are represented by real age-sex (frequency) distributions, since the population size of each sex is included in the clustering process.…”

Symbolic Data Analysis: another look at the interaction of Data Mining and Statistics

“…In Ref , the authors use the Mahalanobis–Wasserstein distance to define a new k ‐means‐type method for histogram‐valued data. Hierarchical clustering based on classical aggregation indices has been addressed in Ref and, in Ref , the authors propose an extension of the Ward method. Irpino and Verde successfully used the Mahalanobis–Wasserstein distance for hierarchical clustering.…”