Ubiquitous Distributed Deep Reinforcement Learning at the Edge: Analyzing Byzantine Agents in Discrete Action Spaces

Zhao, Wenshuai; Queralta, Jorge Peña; Li, Qingqing; Westerlund, Tomi

doi:10.1016/j.procs.2020.10.043

Cited by 8 publications

(4 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have chosen for this study, however, the more recent NVIDIA Isaac Sim platform owing to the high-quality visuals but also tools that enable seamless generation of randomized environments for synthetic data acquisition. Randomization and the ability to alter the environments has been shown to be a key parameter to collaborative learning approaches [15], [20]. An additional advantage is a common ecosystem of tools with the embedded NVIDIA Jetson platforms, the state-of-theart in embedded computing for robots that need discrete GPUs for DL inference.…”

Section: Photorealistic Simulation Platformsmentioning

confidence: 99%

Federated Learning for Vision-based Obstacle Avoidance in the Internet of Robotic Things

Yu¹,

Queralta²,

Westerlund³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Deep learning methods have revolutionized mobile robotics, from advanced perception models for an enhanced situational awareness to novel control approaches through reinforcement learning. This paper explores the potential of federated learning for distributed systems of mobile robots enabling collaboration on the Internet of Robotic Things. To demonstrate the effectiveness of such an approach, we deploy wheeled robots in different indoor environments. We analyze the performance of a federated learning approach and compare it to a traditional centralized training process with a priori aggregated data. We show the benefits of collaborative learning across heterogeneous environments and the potential for simto-real knowledge transfer. Our results demonstrate significant performance benefits of FL and sim-to-real transfer for visionbased navigation, in addition to the inherent privacy-preserving nature of FL by keeping computation at the edge. This is, to the best of our knowledge, the first work to leverage FL for visionbased navigation that also tests results in real-world settings.

show abstract

Section: Photorealistic Simulation Platformsmentioning

confidence: 99%

Federated Learning for Vision-based Obstacle Avoidance in the Internet of Robotic Things

Yu¹,

Queralta²,

Westerlund³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Multi-robot collaboration and DL for robotics have both played an increasingly important role in multiple robotic applications [4]. However, most of the work to date in learning from real-world experiences, sim-to-real transfer, and continuous learning, has been dedicated to reinforcement learning [3], [11] and robotic manipulation [12]. Within the possibilities to achieve collaborative learning, one of the most straightforward approaches is cloud-based centralized learning [13], with a server where data is aggregated and training occurs at once or in batches, but in an offline manner.…”

Section: Related Workmentioning

confidence: 99%

Towards Lifelong Federated Learning in Autonomous Mobile Robots with Continuous Sim-to-Real Transfer

Yu¹,

Queralta²,

Westerlund³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

The role of deep learning (DL) in robotics has significantly deepened over the last decade. Intelligent robotic systems today are highly connected systems that rely on DL for a variety of perception, control and other tasks. At the same time, autonomous robots are being increasingly deployed as part of fleets, with collaboration among robots becoming a more relevant factor. From the perspective of collaborative learning, federated learning (FL) enables continuous training of models in a distributed, privacy-preserving way. This paper focuses on vision-based obstacle avoidance for mobile robot navigation. On this basis, we explore the potential of FL for distributed systems of mobile robots enabling continuous learning via the engagement of robots in both simulated and real-world scenarios. We extend previous works by studying the performance of different image classifiers for FL, compared to centralized, cloud-based learning with a priori aggregated data. We also introduce an approach to continuous learning from mobile robots with extended sensor suites able to provide automatically labelled data while they are completing other tasks. We show that higher accuracies can be achieved by training the models in both simulation and reality, enabling continuous updates to deployed models.

show abstract

“…The main objective of a multi-agent RL is to obtain the localized policies and maximize the global reward for knowledge sharing on the premise of increased system complexity and computation [38]. In multirobot systems, distributed RL can be leveraged to expose different robots to different environments, or to learn more robust policies in the presence of disturbances [39,40].…”

Section: Federated and Distributed Reinforcement Learningmentioning

confidence: 99%

An Overview of Federated Learning at the Edge and Distributed Ledger Technologies for Robotic and Autonomous Systems

Xianjia,

Queralta,

Heikkonen

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Autonomous systems are becoming inherently ubiquitous with the advancements of computing and communication solutions enabling low-latency offloading and real-time collaboration of distributed devices. Decentralized technologies with blockchain and distributed ledger technologies (DLTs) are playing a key role. At the same time, advances in deep learning (DL) have significantly raised the degree of autonomy and level of intelligence of robotic and autonomous systems. While these technological revolutions were taking place, raising concerns in terms of data security and end-user privacy has become an inescapable research consideration. Federated learning (FL) is a promising solution to privacy-preserving DL at the edge, with an inherently distributed nature by learning on isolated data islands and communicating only model updates. However, FL by itself does not provide the levels of security and robustness required by today's standards in distributed autonomous systems. This survey covers applications of FL to autonomous robots, analyzes the role of DLT and FL for these systems, and introduces the key background concepts and considerations in current research.

show abstract

Ubiquitous Distributed Deep Reinforcement Learning at the Edge: Analyzing Byzantine Agents in Discrete Action Spaces

Cited by 8 publications

References 6 publications

Federated Learning for Vision-based Obstacle Avoidance in the Internet of Robotic Things

Federated Learning for Vision-based Obstacle Avoidance in the Internet of Robotic Things

Towards Lifelong Federated Learning in Autonomous Mobile Robots with Continuous Sim-to-Real Transfer

An Overview of Federated Learning at the Edge and Distributed Ledger Technologies for Robotic and Autonomous Systems

Contact Info

Product

Resources

About