FedCO: Communication-Efficient Federated Learning via Clustering Optimization

Abstract

Federated Learning (FL) provides a promising solution for preserving privacy in learning shared models on distributed devices without sharing local data on a central server. However, most existing work shows that FL incurs high communication costs. To address this challenge, we propose a clustering-based federated solution, entitled Federated Learning via Clustering Optimization (FedCO), which optimizes model aggregation and reduces communication costs. In order to reduce the communication costs, we first divide the participating workers into groups based on the similarity of their model parameters and then select only one representative, the best performing worker, from each group to communicate with the central server. Then, in each successive round, we apply the Silhouette validation technique to check whether each representative is still made tight with its current cluster. If not, the representative is either moved into a more appropriate cluster or forms a cluster singleton. Finally, we use split optimization to update and improve the whole clustering solution. The updated clustering is used to select new cluster representatives. In that way, the proposed FedCO approach updates clusters by repeatedly evaluating and splitting clusters if doing so is necessary to improve the workers’ partitioning. The potential of the proposed method is demonstrated on publicly available datasets and LEAF datasets under the IID and Non-IID data distribution settings. The experimental results indicate that our proposed FedCO approach is superior to the state-of-the-art FL approaches, i.e., FedAvg, FedProx, and CMFL, in reducing communication costs and achieving a better accuracy in both the IID and Non-IID cases.

A Survey of Federated Learning From Data Perspective in the Healthcare Domain: Challenges, Methods, and Future Directions

“…For example, a previous study proposed a clustering optimization method based on FL, in which the author utilized similarity to divide the clients into groups and then choose representative workers that communicate with a server; silhouette validation was used to ensure that the main workers are close to their current cluster. [92]. Depending on the complexity of the calculation and the type of data, there are many different criteria for validating the clustering algorithm, including optimization-and differencelike criteria The following table 4 displays the relationship between criteria and complexity: Two or more skews were used to select a client with a lower degree of non-IID to participate in model training [20] [45]…”

HINTS: Human-Centered Intelligent Realities

“…These all will put the baseline and boost the planned collaboration research with some industrial partners involved in HINTS, e.g., NODA Intelligent Systems. In the second area, one of the pursued research directions involved studying approaches that bring efficiency and robustness to FL settings, as discussed in [29]. In a paper [30], researchers from Theme D have proposed a novel FL model that tries coping with statistically heterogeneous environments by introducing a group-personalized FL method.…”

FLWB: a Workbench Platform for Performance Evaluation of Federated Learning Algorithms

“…Each operator in the pipeline has a property called epochs, which defnes a condition that, if satisfed, enables the execution I I of the operator for the specifc epoch. Using this property, it is possible to defne operators to be executed only in the initialization phase and others in the iteration phase of an ML algorithm, like in [7] and [9].…”

“…We decided to implement the FedCO algorithm proposed in [9] and the baseline FL algorithm, FedAvg [5]. The implementation of FedCo leverages the Client clustering pattern, while the implementation of FedAvg leverages the Client registry pattern.…”