Scalable federated machine learning with FEDn

Ekmefjord, Morgan; Ait-Mlouk, Addi; Alawadi, Sadi; Åkesson, Mattias; Stoyanova, Desislava; Spjuth, Ola; Toor, Salman; Hellander, Andreas

doi:10.1109/ccgrid54584.2022.00065

Cited by 27 publications

(7 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have also shown the compatibility of AdaFedAdam with other local optimizers. Future directions include testing AdaFedAdam in real-world geographically distributed setups for both cross-silo and cross-device settings with production grade open source frameworks( [31,4]).…”

Section: Discussionmentioning

confidence: 99%

Accelerating Fair Federated Learning: Adaptive Federated Adam

Li¹,

Zhang²,

Toor³

et al. 2023

Preprint

View full text Add to dashboard Cite

Federated learning is a distributed and privacy-preserving approach to train a statistical model collaboratively from decentralized data of different parties. However, when datasets of participants are not independent and identically distributed (non-IID), models trained by naive federated algorithms may be biased towards certain participants, and model performance across participants is non-uniform. This is known as the fairness problem in federated learning. In this paper, we formulate fairness-controlled federated learning as a dynamical multi-objective optimization problem to ensure fair performance across all participants. To solve the problem efficiently, we study the convergence and bias of Adam as the server optimizer in federated learning, and propose Adaptive Federated Adam (AdaFedAdam) to accelerate fair federated learning with alleviated bias. We validated the effectiveness, Pareto optimality and robustness of AdaFedAdam in numerical experiments and show that AdaFedAdam outperforms existing algorithms, providing better convergence and fairness properties of the federated scheme.Acceleration techniques for federated learning aim at reducing the communication cost and improving convergence. For instance, momentum-based and adaptive optimization methods such as AdaGrad, Adam, Momentum SGD) have been applied to accelerate the training process [11,28,26]). However, default hyperparameters of adaptive optimizers tuned for centralized training do not tend to perform well in federated settings ([26]). Furthermore, optimal hyperparameters are not generalizable for federated learning, and hyperparameter optimization with e.g. grid search are needed for each specific federated task, which is infeasible due to the expensive (and sometimes unbounded) nature for federated learning.

show abstract

Section: Discussionmentioning

confidence: 99%

Accelerating Fair Federated Learning: Adaptive Federated Adam

Li¹,

Zhang²,

Toor³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…While the last two are fairly close to each other, the table leader does not appear to have a solid distance. PySyft [39] is following alone despite FedML [40], and Fedn [53] being very close behind. This is just an initial observation, but things change when we integrate into the equation the popularity results highlighted in Table 1 and the growth rate outlined in Figure 5.…”

Section: Discussionmentioning

confidence: 99%

FeLebrities: a user-centric assessment of Federated Learning frameworks

Riviera¹,

Galazzo²,

Menegaz³

2023

Preprint

View full text Add to dashboard Cite

<p>Federated Learning (FL) is a new paradigm aiming to solve the data access problem. It is gaining an increasing interest in a variety of research fields, including the Biomedical and Financial environments, where lots of valuable data sources are available but not often directly accessible due to the regulations that protect sensitive information. FL provides a solution by moving the focus from sharing data to sharing models. The FL paradigm involves different entities (institutions) holding proprietary datasets, contributing with each other to train a global Artificial Intelligence (AI) model using their own locally available data. Although several studies propose ways to distribute the computation or aggregate results, fewer efforts have been made on how to implement it. With the ambition of helping accelerate the exploitation of FL frameworks, this paper proposes a survey of public tools that are currently available for building FL pipelines, an objective ranking based on the current state of user preferences, and the assessment of the growing trend of the tool’s popularity over a six months time window. Finally, a ranking of the maturity of the tools is derived based on keyaspects to consider when building an FL pipeline. </p>

show abstract

“…The federated models are trained using FEDn [50], a production-grade federated learning framework deployed on NAISS cloud [51], UPPMAX [52] and Alvis [53] High-Performance Computing (HPC) clusters in a distributed manner. The architecture of the training framework is shown in Figure 3.…”

Section: Methodsmentioning

confidence: 99%

Federated Learning for Predicting Compound Mechanism of Action Based on Image-data from Cell Painting

Ju,

Hellander,

Spjuth

2024

Preprint

Self Cite

View full text Add to dashboard Cite

Having access to sufficient data is essential in order to train accurate machine learning models, but much data is not publicly available. In drug discovery this is particularly evident, as much data is withheld at pharmaceutical companies for various reasons. Federated Learning (FL) aims at training a joint model between multiple parties but without disclosing data between the parties. In this work, we leverage Federated Learning to predict compound Mechanism of Action (MoA) using fluorescence image data from cell painting. Our study evaluates the effectiveness and efficiency of FL, comparing to non-collaborative and data-sharing collaborative learning in diverse scenarios. Specifically, we investigate the impact of data heterogeneity across participants on MoA prediction, an essential concern in real-life applications of FL, and demonstrate the benefits for all involved parties. This work highlights the potential of federated learning in multi-institutional collaborative machine learning for drug discovery and assessment of chemicals, offering a promising avenue to overcome data-sharing constraints.

show abstract

Scalable federated machine learning with FEDn

Cited by 27 publications

References 13 publications

Accelerating Fair Federated Learning: Adaptive Federated Adam

Accelerating Fair Federated Learning: Adaptive Federated Adam

FeLebrities: a user-centric assessment of Federated Learning frameworks

Federated Learning for Predicting Compound Mechanism of Action Based on Image-data from Cell Painting

Contact Info

Product

Resources

About