Decentralized and Adaptive K-Means Clustering for Non-IID Data Using HyperLogLog Counters

Abstract

The data shared over the Internet tends to originate from ubiquitous and autonomous sources such as mobile phones, fitness trackers, and IoT devices. Centralized and federated machine learning solutions represent the predominant way of providing smart services for users. However, moving data to central location for analysis causes not only many privacy concerns, but also communication overhead. Therefore, in certain situations machine learning models need to be trained in a collaborative and decentralized manner, similar to the way the data is originally generated without requiring any central authority for data or model aggregation. This paper presents a decentralized and adaptive k -means algorithm that clusters data from multiple sources organized in peer-topeer networks. Our algorithm allows peers to reach an approximation of the global model without sharing any raw data. Most importantly, we address the challenge of decentralized clustering with skewed non-IID data and asynchronous computations by integrating HyperLogLog counters with k -means algorithm. Furthermore, our clustering algorithm allows nodes to individually determine the number of clusters that fits their local data. Results using synthetic and real-world datasets show that our algorithm outperforms state-of-the-art decentralized k -means algorithms achieving accuracy gain that is up-to 36%.

Efficient Privacy Preserving Distributed K-Means for Non-IID Data

“…An alternative private distributed clustering approach is to select a subset of local points (representatives) and apply clustering over them. Soliman et al [21] proposed running the K-Means algorithm locally on each client and using HyperLogLog counters to share the centroids and the approximate number of observations per centroid in a decentralized fashion with the other clients. Then a weighted averaging over all the centroids is done to find the global centroids.…”

Federated Learning and Differential Privacy: Software tools analysis, the Sherpa.ai FL framework and methodological guidelines for preserving data privacy

“…• To aggregate the information learned from each of the clients into a global model using classical federated aggregation operators such as: FedAvg, weighted FedAvg [24] and an aggregation operator for the adaptation of the k-means algorithm to a federated setting [25].…”

Evaluation Mechanism for Decentralized Collaborative Pattern Learning in Heterogeneous Vehicular Networks

“…Although there are many works concentrating on the distributed learning with clustering as methods to compute local model [17,18,19,20,21], few works have considered the performance aspects of ML models in an inherently distributed network, especially when there are sub-groups of agents receiving data from different phenomena. At some timestamps T i , some agents may have learned their provisional global models or received model from their neighbours.…”