Federated Reinforcement Learning for Fast Personalization

Nadiger, Chetan; Kumar, Anil; Abdelhak, Sherine

doi:10.1109/aike.2019.00031

Cited by 73 publications

(42 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is a few federated reinforcement learning methods based on parameter sharing. [17] introduces a communication policy between the server and clients, which decides whether to update the global model on the server or share the global model to clients. [18], [19] shares both the gradient and policy parameters with other clients to calculate the federated RL algorithm.…”

Section: B Federated Reinforcement Learningmentioning

confidence: 99%

“…FL can be naturally introduced into RL to implement secure cooperative training in RL algorithms. However, some of the existing federated reinforcement learning methods are algorithm-independent but need directly sharing parameters of policy [17]- [22]. which will cause revealing privacy while others are limited to some specific scenarios [23], [24].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Reward Shaping Based Federated Reinforcement Learning

Hua

Liu

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Federated reinforcement learning aims to promote training efficiency or improve policy quality through information interaction with privacy protection. Existing federated reinforcement learning methods rarely utilize the structure of reinforcement learning algorithms while are limiting to specific scenarios or algorithms. We propose a general federated reinforcement learning framework FRS, which employs reward shaping as the federated information shared among different clients with different tasks to promote each client's training speed and policy quality. The federated reward shaping is implicitly learned by average state value information of all clients to protect each client's task privacy as the real trajectory is anonymous. Experiments on the GridWorld environment show that FRS can algorithm-independently improve the policy quality and promote training speed with protecting each client's privacy.

show abstract

Section: B Federated Reinforcement Learningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Reward Shaping Based Federated Reinforcement Learning

Hua

Liu

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Alternatively, during the training process, the vehicles could share their trained models with each other periodically to enhance the training process and obtain a better trained agent in less amount of time. In order to do so, we investigate the role of federated RL [14]. FL is a machine learning framework where different clients (vehicles) collaboratively train a model under the orchestration of a central entity (RSU), while keeping the training data (experience) decentralized and private [10].…”

Section: A Federated Rlmentioning

confidence: 99%

Cooperative perception in Vehicular Networks using Multi-Agent Reinforcement Learning

Abdel-Aziz

Samarakoon

Perfecto

et al. 2020

2020 54th Asilomar Conference on Signals, Systems, and Computers

View full text Add to dashboard Cite

Cooperative perception plays a vital role in extending a vehicle's sensing range beyond its line-of-sight. However, exchanging raw sensory data under limited communication resources is infeasible. Towards enabling an efficient cooperative perception, vehicles need to address fundamental questions such as: what sensory data needs to be shared? at which resolution? In this view, this paper proposes a reinforcement learning (RL)based content selection of cooperative perception messages by utilizing a quadtree-based point cloud compression mechanism. Furthermore, we investigate the role of federated RL to enhance the training process. Simulation results show the ability of the RL agents to efficiently learn the message content selection that maximizes the satisfaction of the vehicles in terms of the received sensory information. It is also shown that federated RL improves the training process, where better policies can be achieved within the same amount of time compared to the nonfederated approach.

show abstract

“…Furthermore, for applying this FL concept to various engineering control problems, federated reinforcement learning (FRL) [ 31 ] was proposed wherein the optimal policy for individual agent was calculated as long as ensuring that the data were not shared among agents during the training process. The FRL method was employed for developing the navigation for cloud-based robotic systems [ 32 ], rapid personalization process of agents [ 33 ], and as a defense strategy for jamming attacks in the flying ad hoc network for unmanned aerial vehicles [ 34 ]. FRL has been adopted to resolve distributed control problems in a variety of engineering fields.…”

Section: Introductionmentioning

confidence: 99%

Privacy-Preserving Energy Management of a Shared Energy Storage System for Smart Buildings: A Federated Deep Reinforcement Learning Approach

Lee

Xie

Choi

2021

Sensors

View full text Add to dashboard Cite

This paper proposes a privacy-preserving energy management of a shared energy storage system (SESS) for multiple smart buildings using federated reinforcement learning (FRL). To preserve the privacy of energy scheduling of buildings connected to the SESS, we present a distributed deep reinforcement learning (DRL) framework using the FRL method, which consists of a global server (GS) and local building energy management systems (LBEMSs). In the framework, the LBEMS DRL agents share only a randomly selected part of their trained neural network for energy consumption models with the GS without consumer’s energy consumption data. Using the shared models, the GS executes two processes: (i) construction and broadcast of a global model of energy consumption to the LBEMS agents for retraining their local models and (ii) training of the SESS DRL agent’s energy charging and discharging from and to the utility and buildings. Simulation studies are conducted using one SESS and three smart buildings with solar photovoltaic systems. The results demonstrate that the proposed approach can schedule the charging and discharging of the SESS and an optimal energy consumption of heating, ventilation, and air conditioning systems in smart buildings under heterogeneous building environments while preserving the privacy of buildings’ energy consumption.

show abstract

Federated Reinforcement Learning for Fast Personalization

Cited by 73 publications

References 6 publications

Reward Shaping Based Federated Reinforcement Learning

Reward Shaping Based Federated Reinforcement Learning

Cooperative perception in Vehicular Networks using Multi-Agent Reinforcement Learning

Privacy-Preserving Energy Management of a Shared Energy Storage System for Smart Buildings: A Federated Deep Reinforcement Learning Approach

Contact Info

Product

Resources

About