FedKL: Tackling Data Heterogeneity in Federated Reinforcement Learning by Penalizing KL Divergence

Xie, Zhixun; Song, S. H.

doi:10.48550/arxiv.2204.08125

Cited by 2 publications

(2 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…New insights for protecting against the non-uniformity introduced by data heterogeneity in FL as a solution for backdooring attacks were shown in [145]. FedKL [146] used federated reinforcement learning to tackle heterogeneity issues. Data problems and resource heterogeneity in FL were discussed in [147].…”

Section: ) Research On Handling Corrupted or Noisy Clients In Flmentioning

confidence: 99%

Decentralized Learning in Healthcare: A Review of Emerging Techniques

2023

View full text Add to dashboard Cite

Recent developments in deep learning have contributed to numerous success stories in healthcare. The performance of a deep learning model generally improves with the size of the training data. However, there are privacy, ownership, and regulatory issues that prevent combining medical data into traditional centralized storage. Decentralized learning approaches enable collaborative model training by distributing the learning process among several nodes or devices. Conceptually, decentralized learning builds on earlier work in distributed optimization, but the focus of this paper is on recent and emerging techniques such as Federated Learning (FL), Split Learning (SL), and hybrid Split-Federated Learning (SFL). With common, universal deep learning models and centralized aggregator servers, FL overcomes the difficulties of centralized training. Additionally, patient data remains at the local party, upholding the security and anonymity of the data. SL enables machine learning without directly accessing data on clients or end devices. It further enhances privacy in a decentralized setting and mitigates clients' storage issues. In this survey, we first provide a contemporary survey of FL, SL, and SFL approaches. Second, we discuss their state-ofthe-art applications in healthcare, particularly in medical image analysis. Third, we review these emerging decentralized learning approaches under challenging conditions such as statistical and system heterogeneity, privacy preservation, communication efficiency, fairness, etc. Then, we address existing approaches to tackle these challenges. We detail unique complications related to healthcare applications including data, privacy and security, and communication challenges. Finally, we outline potential areas for further research on emerging decentralized learning techniques in healthcare, including developing personalized models, reducing bias, incorporating hybrid non-IID features, hyperparameter tuning, developing sufficient incentive mechanisms, and incorporating domain expertise knowledge.

show abstract

Section: ) Research On Handling Corrupted or Noisy Clients In Flmentioning

confidence: 99%

Decentralized Learning in Healthcare: A Review of Emerging Techniques

2023

View full text Add to dashboard Cite

show abstract

“…The authors define heterogeneity as different state transition functions among siloed clients in the federated system, while for each client the environment is homogeneous. In another more recent study by Xie & Song (2022), heterogeneity of initial state distribution and heterogeneity of environment dynamics are both taken into consideration. Our problem is also closely related to Hidden Parameter Markov Decision Processes (HiP-MDPs) (Doshi-Velez & Konidaris, 2016) which consider a set of closely related MDPs which can be fully specified with a bounded number of latent parameters.…”

Section: Related Workmentioning

confidence: 99%

Stateful active facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learning

Liu¹,

Shah²,

Boussif³

et al. 2022

Preprint

View full text Add to dashboard Cite

In cooperative multi-agent reinforcement learning, a team of agents works together to achieve a common goal. Different environments or tasks may require varying degrees of coordination among agents in order to achieve the goal in an optimal way. The nature of coordination will depend on properties of the environment-its spatial layout, distribution of obstacles, dynamics, etc. We term this variation of properties within an environment as heterogeneity. Existing literature has not sufficiently addressed the fact that different environments may have different levels of heterogeneity. We formalize the notions of coordination level and heterogeneity level of an environment and present HECOGrid, a suite of multi-agent RL environments that facilitates empirical evaluation of different MARL approaches across different levels of coordination and environmental heterogeneity by providing a quantitative control over coordination and heterogeneity levels of the environment. Further, we propose a Centralized Training Decentralized Execution learning approach called Stateful Active Facilitator (SAF) that enables agents to work efficiently in high-coordination and high-heterogeneity environments through a differentiable and shared knowledge source used during training and dynamic selection from a shared pool of policies. We evaluate SAF and compare its performance against baselines IPPO and MAPPO on HECOGrid. Our results show that SAF consistently outperforms the baselines across different tasks and different heterogeneity and coordination levels.

show abstract

FedKL: Tackling Data Heterogeneity in Federated Reinforcement Learning by Penalizing KL Divergence

Cited by 2 publications

References 13 publications

Decentralized Learning in Healthcare: A Review of Emerging Techniques

Decentralized Learning in Healthcare: A Review of Emerging Techniques

Stateful active facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learning

Contact Info

Product

Resources

About