Scalable Perception-Action-Communication Loops With Convolutional and Graph Neural Networks

Hu, Ting-Kuei; Gama, Fernando; Chen, Tianlong; Zheng, Wenqing; Wang, Zhangyang; Ribeiro, Alejandro; Sadler, Brian M.

doi:10.1109/tsipn.2021.3139336

Cited by 13 publications

(13 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[• • 7]), where resilience can be provided by generalisation or by learning on a newly provided data set. The source of the data set varies but common examples are simulated trajectories generated by centralised expert controllers which have global state information [7,25] and real-world video recordings of humans performing the desired behaviour [26]. In robust-by-design, each control cycle accounts for a range of environmental changes, for example by implicitly communicating occupancy map changes [9] or by explicitly communicating the availability and capability of team members [27].…”

Section: Adaptation To Changesmentioning

confidence: 99%

“…Model-based fault-detection [16,28,29] Perception-actioncommunication loops [7,25] Independent Dec-POMDPs [46] Networking Dec-POMDPs [47,48] Transition dynamics or task requirements…”

Section: Communication Communication Noise Local Communication Rangementioning

confidence: 99%

“…Incorporating perception-action-communication loops into graph neural networks (GNNs) is a recent solution to decentralised control and learning-how-tocommunicate [7,25]. In such works, the GNN takes as input a communication graph, which represents robots as nodes, communication links as edges, and communication delays as their distances.…”

Section: Learning Perception-action-communication Loopsmentioning

confidence: 99%

“…Decentralised control in this sense has been studied for multirobot systems [5,6], which have fewer robots with more powerful hardware and high-complexity algorithms such as deep neural networks (e.g. [7]), and swarm robotics [8], which is scalable in team size but comes with simplistic hardware and low-complexity algorithms such as ant-inspired algorithms (e.g. [9]).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Resilient robot teams: a review integrating decentralised control, change-detection, and learning

Bossens,

Ramchurn,

Tarapore

2022

Preprint

View full text Add to dashboard Cite

Purpose of review: This paper reviews opportunities and challenges for decentralised control, change-detection, and learning in the context of resilient robot teams. Recent findings: Exogenous fault detection methods can provide a generic detection or a specific diagnosis with a recovery solution. Robot teams can perform active and distributed sensing for detecting changes in the environment, including identifying and tracking dynamic anomalies, as well as collaboratively mapping dynamic environments. Resilient methods for decentralised control have been developed in learning perception-action-communication loops, multi-agent reinforcement learning, embodied evolution, offline evolution with online adaptation, explicit task allocation, and stigmergy in swarm robotics. Summary: Remaining challenges for resilient robot teams are integrating change-detection and trial-and-error learning methods, obtaining reliable performance evaluations under constrained evaluation time, improving the safety of resilient robot teams, theoretical results demonstrating rapid adaptation to given environmental perturbations, and designing realistic and compelling case studies.

show abstract

Section: Adaptation To Changesmentioning

confidence: 99%

“…Model-based fault-detection [16,28,29] Perception-actioncommunication loops [7,25] Independent Dec-POMDPs [46] Networking Dec-POMDPs [47,48] Transition dynamics or task requirements…”

Section: Communication Communication Noise Local Communication Rangementioning

confidence: 99%

Section: Learning Perception-action-communication Loopsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Resilient robot teams: a review integrating decentralised control, change-detection, and learning

Bossens,

Ramchurn,

Tarapore

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In cases where the downstream task is tightly coupled with the communication requirements, it is beneficial to optimize the communication strategy jointly with perception and action policies. This was done in [134], for multi-robot flocking, and in [133], for multi-agent path planning. These frameworks implement a cascade of a convolutional neural network (CNN) and a GNN, which they jointly train so that image features and communication messages are learned in conjunction to better address the specific task.…”

Section: Learning Communication Behaviorsmentioning

confidence: 99%

A Critical Review of Communications in Multi-Robot Systems

Gielis¹,

Singh²,

Prorok³

2022

Preprint

View full text Add to dashboard Cite

Purpose of Review. This review summarizes the broad roles that communication formats and technologies have played in enabling multi-robot systems. We approach this field from two perspectives: of robotic applications that need communication capabilities in order to accomplish tasks, and of networking technologies that have enabled newer and more advanced multi-robot systems. Recent Findings. Through this review, we identify a dearth of work that holistically tackles the problem of co-design and co-optimization of robots and the networks they employ. We also highlight the role that data-driven and machinelearning approaches play in evolving communication pipelines for multi-robot systems. In particular, we refer to recent work that diverges from hand-designed communication patterns, and also discuss the 'sim-to-real' gap in this context. Summary. We present a critical view of the way robotic algorithms and their networking systems have evolved, and make the case for a more synergistic approach. Finally, we also identify four broad Open Problems for research and development, while offering a data-driven perspective for solving some of them.

show abstract

Resilient Robot Teams: a Review Integrating Decentralised Control, Change-Detection, and Learning

2022

View full text Add to dashboard Cite

Purpose of Review This paper reviews opportunities and challenges for decentralised control, change-detection, and learning in the context of resilient robot teams. Recent Findings Exogenous fault-detection methods can provide a generic detection or a specific diagnosis with a recovery solution. Robot teams can perform active and distributed sensing for detecting changes in the environment, including identifying and tracking dynamic anomalies, as well as collaboratively mapping dynamic environments. Resilient methods for decentralised control have been developed in learning perception-action-communication loops, multi-agent reinforcement learning, embodied evolution, offline evolution with online adaptation, explicit task allocation, and stigmergy in swarm robotics. Summary Remaining challenges for resilient robot teams are integrating change-detection and trial-and-error learning methods, obtaining reliable performance evaluations under constrained evaluation time, improving the safety of resilient robot teams, theoretical results demonstrating rapid adaptation to given environmental perturbations, and designing realistic and compelling case studies.

show abstract

Scalable Perception-Action-Communication Loops With Convolutional and Graph Neural Networks

Cited by 13 publications

References 34 publications

Resilient robot teams: a review integrating decentralised control, change-detection, and learning

Resilient robot teams: a review integrating decentralised control, change-detection, and learning

A Critical Review of Communications in Multi-Robot Systems

Resilient Robot Teams: a Review Integrating Decentralised Control, Change-Detection, and Learning

Contact Info

Product

Resources

About