Clustered Federated Learning in Heterogeneous Environment

Yan, Yihan; Tong, Xiaojun; Wang, Shen

doi:10.1109/tnnls.2023.3264740

Cited by 7 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Presotto et al [32] proposed a federated clustering algorithm FedCLAR, which grouped clients based on the similarity of client models, so as to better identify and distinguish client data with different distributions. Yan et al [12] proposed ICFL, which can dynamically determine the cluster structure of clients during each training round and aggregate a personalized model for each cluster. Long et al [15] proposed multicenter FL, which forms multiple personalized models through clustering based on distributed similarity of data and optimizes them individually.…”

Section: Personalized Federated Learningmentioning

confidence: 99%

“…ICFL [12], a clustering FL algorithm that automatically clusters clients and aggregates clustering models according to the correlation between clients without setting the number of clusters. 4.…”

Section: Set Upmentioning

confidence: 99%

“…In response to the above problems, Personalized Federated Learning (PFL) was proposed to learn a personalized model for the client that has better performance on local data while still benefiting from collaborative training. One way to implement PFL is to cluster clients with similar data distributions, and the model parameters of similar clients are aggregated as their personalized models [12][13][14][15][16][17][18][19]. For example, Yan et al [12] proposed ICFL, which can dynamically determine the cluster structure of clients during each training round and aggregate a personalized model for each cluster.…”

Section: Introductionmentioning

confidence: 99%

“…One way to implement PFL is to cluster clients with similar data distributions, and the model parameters of similar clients are aggregated as their personalized models [12][13][14][15][16][17][18][19]. For example, Yan et al [12] proposed ICFL, which can dynamically determine the cluster structure of clients during each training round and aggregate a personalized model for each cluster. In [13], Wang et al proposed CPFL, which uses Earth mover's distance to measure the distribution similarity between client data, thereby clustering clients to combat the Non-IID problem.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

FedScrap: Layer-Wise Personalized Federated Learning for Scrap Detection

Zhang,

Deng,

Wang

2024

Electronics

View full text Add to dashboard Cite

Scrap steel inspection is a critical entry point for connecting the smelting process to the industrial internet, with its security and privacy being of vital importance. Current advancements in scrap steel inspection involve collecting scattered data through the industrial internet, then utilizing them to train machine learning models for distributed classification. However, this detection method exposes original scrap steel data directly to the industrial internet, making it susceptible to interception by attackers, who can potentially obtain sensitive information. This paper presents a layer-wise personalized federated framework for scrap steel detection, termed FedScrap, which leverages federated learning (FL) to coordinate decentralized and heterogeneous scrap steel data while ensuring data privacy protection. The key challenge that FedScrap confronts is the heterogeneity of scrap steel data distributed across the network, which complicates the task of effectively integrating these data into a single detection model constructed via FL. To address this challenge, FedScrap employs a self-attention mechanism to aggregate personalized models for each layer of every client, focusing on the most relevant models to their specific data. By assigning higher attention scores to more relevant models, it achieves more accurate aggregation weights during the model aggregation process. To validate the efficacy of the proposed method, a dataset of scrap images was collected from a steel mill, and the results demonstrate that FedScrap achieves accurate classification of distributed scrap data with an impressive accuracy rate of 90%.

show abstract

Section: Personalized Federated Learningmentioning

confidence: 99%

Section: Set Upmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

FedScrap: Layer-Wise Personalized Federated Learning for Scrap Detection

Zhang,

Deng,

Wang

2024

Electronics

View full text Add to dashboard Cite

show abstract

Accelerating Federated Learning via Sequential Training of Grouped Heterogeneous Clients

Silvi,

Rizzardi,

Caldarola

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Federated Learning (FL) allows training machine learning models in privacy-constrained scenarios by enabling the cooperation of edge devices without requiring local data sharing. This approach raises several challenges due to the different statistical distribution of the local datasets and the clients' computational heterogeneity. In particular, the presence of highly non-i.i.d. data severely impairs both the performance of the trained neural network and its convergence rate, increasing the number of communication rounds required to reach centralized performance. As a solution, we propose FedSeq, a novel framework leveraging the sequential training of subgroups of heterogeneous clients, i.e., superclients, to learn more robust models before the server-side averaging step. Given a fixed budget of communication rounds, we show that FedSeq outperforms or match several state-of-the-art federated algorithms in terms of final performance and speed of convergence. Our method can be easily integrated with other approaches available in the literature, and empirical results show that combining existing algorithms with FedSeq further improves its final performance and convergence speed. We evaluate our method across multiple FL benchmarks, establishing its effectiveness in both i.i.d. and non-i.i.d. scenarios. Lastly, we highlight that the sequential training introduced here does not introduce additional privacy concerns when compared to the de facto standard, FedAvg.

show abstract