Federated Learning with Non-Iid Data Via Local Model Pruning

huang, youx; Zhu, Shunzhi; Huang, Zhicai

doi:10.2139/ssrn.4411404

Cited by 2 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the i.i.d data setting, data distribution over UEs is balanced and uniform. For the non-i.i.d data setting, the dataset distribution over UEs is unbalanced, where the unbalanced feature means that the dataset size varies greatly between different UEs [37]. In [37]- [39], the performance of FL systems is very sensitive to non-i.i.d data distributions and experiments for the non-i.i.d data setting could evaluate the robustness of the proposed algorithm.…”

Section: A Experimental Settingsmentioning

confidence: 99%

“…For the non-i.i.d data setting, the dataset distribution over UEs is unbalanced, where the unbalanced feature means that the dataset size varies greatly between different UEs [37]. In [37]- [39], the performance of FL systems is very sensitive to non-i.i.d data distributions and experiments for the non-i.i.d data setting could evaluate the robustness of the proposed algorithm. To this end, we take experiments with different datasets for the non-i.i.d data setting to verify the robustness of the proposed MOEA-FL.…”

Section: A Experimental Settingsmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Objective Optimization for Bandwidth-Limited Federated Learning in Wireless Edge Systems

Zhou

Liu

Lei

2023

IEEE Open J. Commun. Soc.

View full text Add to dashboard Cite

This paper studies a bandwidth-limited federated learning (FL) system where the access point is a central server for aggregation and the energy-constrained user equipemnts (UEs) with limited computation capabilities (e.g., Internet of Things devices) perform local training. Limited by the bandwidth in wireless edge systems, only a part of UEs can participate in each FL training round. Selecting different UEs could affect the FL performance, and selected UEs need to allocate their computing resource effectively. In wireless edge FL systems, simultaneously accelerating FL training and reducing computingcommunication energy consumption are of importance. To this end, we formulate a multi-objective optimization problem (MOP). In MOP, the model training convergence is difficult to calculate accurately. Meanwhile, MOP is a combinatorial optimization problem, with the high-dimension mix-integer variables, which is proved to be NP-hard. To address these challenges, a multi-objective evolutionary algorithm for the bandwidth-limited FL system (MOEA-FL) is proposed to obtain a Pareto optimal solution set. In MOEA-FL, an age-of-update-loss method is first proposed to transform the original global loss function into a convergence reference function. Then, MOEA-FL divides MOP into N single objective subproblems by the Tchebycheff approach and optimizes the subproblems simultaneously by evolving a population. Extensive experiments have been carried out on MNIST dataset and a medical case called TissueMNIST dataset for both the i.i.d and non-i.i.d data setting. Experimental results demonstrate that MOEA-FL performs better than other algorithms and verify the robustness and scalability of MOEA-FL.

show abstract

Section: A Experimental Settingsmentioning

confidence: 99%

Section: A Experimental Settingsmentioning

confidence: 99%

Multi-Objective Optimization for Bandwidth-Limited Federated Learning in Wireless Edge Systems

Zhou

Liu

Lei

2023

IEEE Open J. Commun. Soc.

View full text Add to dashboard Cite

show abstract

“…These techniques can reduce communication overhead and enable the system to scale to a larger number of clients. A variety of approaches have been proposed and their potential has been demonstrated in several studies [23,29,44,[57][58][59].…”

Section: Data Decoupling Techniques In Federated Learningmentioning

confidence: 99%

Comparative Evaluation of Data Decoupling Techniques for Federated Machine Learning with Database as a Service

Khan¹,

Hu²,

Sadoghi³

et al. 2023

Preprint

View full text Add to dashboard Cite

Federated learning (FL) is a distributed machine learning approach that allows multiple clients to learn a shared model collaboratively without sharing their raw data. State-of-the-art FL systems provide an all-in-one solution; for example, users must install and use whichever data management subsystem that is expected by the FL system. This tightly-coupling paradigm is at best a hindrance to the wide adoption of FL solutions; in some domains, such as scientific applications, it could be a deal-breaker by enforcing a specific type of data solution on special hardware and platforms (e.g., high-performance computing clusters without node-local persistent storage). To this end, one natural solution is to decouple the data management functionalities from the FL system, enabling clients to customize their FL applications with specific data subsystems on which efficient queries can be made. The technical challenges of such decoupling methods include new system architectures, expressive and structured data models for FL parameters, and security guarantees among FL participants. As a starting point for this line of research, this paper conducts a thorough evaluation and comparison of mainstream database solutions as decoupled services in the context of FL systems. To make a fair comparison, we develop a framework called Data-Decoupling Federated Learning (DDFL), which can run FL workloads and is agnostic of the underlying database system by exposing a unified interface. To evaluate the effectiveness and feasibility of our approach, we compare DDFL with state-of-the-art FL systems that tightly couple data management and computational counterparts. We carry out extensive experiments on various datasets and data management subsystems (e.g., PostgreSQL, MongoDB, Cassandra, Neo4j) and show that DDFL achieves comparable or better performance in terms of training time, inference accuracy, and database query time, while providing more options for clients to tune their FL applications regarding data-related metrics, such as performance, resilience, and usability. Additionally, we provide a detailed qualitative analysis of DDFL when being integrated with mainstream database systems.

show abstract

Federated Learning with Non-Iid Data Via Local Model Pruning

Cited by 2 publications

References 0 publications

Multi-Objective Optimization for Bandwidth-Limited Federated Learning in Wireless Edge Systems

Multi-Objective Optimization for Bandwidth-Limited Federated Learning in Wireless Edge Systems

Comparative Evaluation of Data Decoupling Techniques for Federated Machine Learning with Database as a Service

Contact Info

Product

Resources

About