Efficient Privacy Preserving Distributed K-Means for Non-IID Data

Abstract

Privacy is becoming a crucial requirement in many machine learning systems. In this paper we introduce an efficient and secure distributed K-Means algorithm, that is robust to non-IID data. The base idea of our proposal consists in each client computing the K-Means algorithm locally, with a variable number of clusters. The server will use the resultant centroids to apply the K-Means algorithm again, discovering the global centroids. To maintain the client's privacy, homomorphic encryption and secure aggregation is used in the process of learning the global centroids. This algorithm is efficient and reduces transmission costs, since only the local centroids are used to find the global centroids. In our experimental evaluation, we demonstrate that our strategy achieves a similar performance to the centralized version even in cases where the data follows an extreme non-IID form.

Greedy centroid initialization for federated K-means

“…ii) Client Initialization First: Since the initial centroid selection starts closer to the data source, this approach has shown promise in enhancing overall performance. Brandao et al [13] suggested an approach where each client first locally determines the optimal ∼ K within the range [1, K] using the Silhouette metric [14]. Subsequently, they share the best ∼ K initial centroids with the server.…”

“…Dennis et al [17] introduced a one-shot federated clustering scheme based on K-means (known as K-FED). However, a practical drawback shared with Brandao's [13] method is the substantial computational burden placed on edge devices, which typically have constrained computing capabilities. Similar to Dennis et al's [17] approach, we start the initialization for federated K-means at the edge clients, where the data resides.…”

Hybrid Cloud-Based Privacy Preserving Clustering as Service for Enterprise Big Data

“…Negative database-based methods have two problemsconversion to negative database is not possible for all kinds of data and there is huge overhead on data owner side for negative database construction. Brando et al (2021) [18] proposed a distributed privacy preserving K-mean algorithm. Client compute K-mean for their data locally and send the centroids to a server.…”