Simultaneous clustering and variable selection: A novel algorithm and model selection procedure

Abstract

The growing availability of high-dimensional data sets offers behavioral scientists an unprecedented opportunity to integrate the information hidden in the novel types of data (e.g., genetic data, social media data, and GPS tracks, etc.,) and thereby obtain a more detailed and comprehensive view towards their research questions. In the context of clustering, analyzing the large volume of variables could potentially result in an accurate estimation or a novel discovery of underlying subgroups. However, a unique challenge is that the high-dimensional data sets likely involve a significant amount of irrelevant variables. These irrelevant variables do not contribute to the separation of clusters and they may mask cluster partitions. The current paper addresses this challenge by introducing a new clustering algorithm, called Cardinality K-means or CKM, and by proposing a novel model selection strategy. CKM is able to perform simultaneous clustering and variable selection with high stability. In two simulation studies and an empirical demonstration with genetic data, CKM consistently outperformed competing methods in terms of recovering cluster partitions and identifying signaling variables. Meanwhile, our novel model selection strategy determines the number of clusters based on a subset of variables that are most likely to be signaling variables. Through a simulation study, this strategy was found to result in a more accurate estimation of the number of clusters compared to the conventional strategy that utilizes the full set of variables. Our proposed CKM algorithm, together with the novel model selection strategy, has been implemented in a freely accessible R package.

Optimizing Urban-Scale Evacuation Strategies Through Disaster Victim Aggregation Modification

“…The methodology determining the optimal combination of K-means and DBSCAN can follow two methods. The first method involves searching all possible combinations of K and e sequentially to find the most effective configuration [67]. However, this method requires extensive resources and may be impractical during time-sensitive disasters.…”

Integrating Unsupervised and Supervised ML Models for Analysis of Synthetic Data From VAE, GAN, and Clustering of Variables

“…However, as the applications of ML algorithms expand in scope, and with the advent of wearable devices and other technological advances (Tufail et al, 2023), researchers confront two main challenges: (1) obtain a sufficiently large data set necessary for constructing meaningful machine learning models (L'Heureux et al, 2017) and (2) high-dimensional data sets are becoming more prevalent across multiple disciplines (Yuan, 2023) including genetics (Chi et al, 2016), organizational psychology, and neuroscience (Waldman et al, 2019). The effectiveness of machine learning models is inherently tied to the quality and quantity of data used for training.…”