Stability estimation for unsupervised clustering: A review

Abstract

Cluster analysis remains one of the most challenging yet fundamental tasks in unsupervised learning. This is due in part to the fact that there are no labels or gold standards by which performance can be measured. Moreover, the wide range of clustering methods available is governed by different objective functions, different parameters, and dissimilarity measures. The purpose of clustering is versatile, often playing critical roles in the early stages of exploratory data analysis and as an endpoint for knowledge and discovery. Thus, understanding the quality of a clustering is of critical importance. The concept of stability has emerged as a strategy for assessing the performance and reproducibility of data clustering. The key idea is to produce perturbed data sets that are very close to the original, and cluster them. If the clustering is stable, then the clusters from the original data will be preserved in the perturbed data clustering. The nature of the perturbation, and the methods for quantifying similarity between clusterings, are nontrivial, and ultimately what distinguishes many of the stability estimation methods apart. In this review, we provide an overview of the very active research area of cluster stability estimation and discuss some of the open questions and challenges that remain in the field. This article is categorized under:Statistical Learning and Exploratory Methods of the Data Sciences > Clustering and Classification

Multi-view clustering by CPS-merge analysis with application to multimodal single-cell data

“…The tightness measure is defined in [ 40 ], which quantifies the clustering stability. A comprehensive review of clustering stability is referred to [ 41 ]. If the number of Cartesian product clusters at the start of merging is large (for example, more than 100), we conduct a first-stage merging by bipartite clustering.…”

A Review on Appropriateness of Partitional Clustering Algorithms in Handling Transactional Data

“…As one of unsupervised learning approach, clustering is a fundamental concept in entailing the exploration and organization of data without the aid of labelled outcomes. In this approach, the algorithm endeavors to reveal concealed patterns and relationships within the dataset by grouping similar data points into distinct clusters (Liu et al, 2022). Furthermore, clustering is frequently used in any discipline that involves multivariate data analysis (Lipovetsky, 2022).…”

Clustering Deviation Index (CDI): a robust and accurate internal measure for evaluating scRNA-seq data clustering

“…In addition to internal and external indices, some stability indices evaluate clustering methods based on the clustering stability when the original data are perturbed (resampled, split, subsampled). See [21] for a recent review of these indices. The key idea is to produce perturbed datasets whose distribution is close to the original and apply the clustering methods.…”