Scaling Up Multiagent Reinforcement Learning for Robotic Systems: Learn an Adaptive Sparse Communication Graph

Shen, Macheng; How, Jonathan P.

doi:10.1109/iros45743.2020.9341303

Cited by 12 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The utilisation of GNNs can change according to the information that are encoded in the graph. Many notable works [43], [134]- [143] focus the attention on the communication between robots, a crucial aspect for task and motion planning, and exploit graph encoding to efficiently model inter-agents relationships. Usually, graph neural networks are used to learn a communication graph, which is then exploited for example to directly support the agents in making decisions, or to foster multi-agent reinforcement learning frameworks in policy learning.…”

Section: Gnns For Multi-agent Systemsmentioning

confidence: 99%

“…In section IV, we reviewed several approaches that build a communication graph for multi-robot systems coordination, which can be also applied for exploration tasks [43], [134]- [142]. Some of these works, instead, are specifically designed for the task of exploration, such as [142] where the authors tackle the task of coverage control, with the aim of predicting the distribution of a set of robots in a region such that the likelihood to spot events of interest is maximised.…”

Section: B Exploration and Navigationmentioning

confidence: 99%

See 1 more Smart Citation

Graph Learning in Robotics: A Survey

Pistilli,

Averta

2023

IEEE Access

View full text Add to dashboard Cite

Deep neural networks for graphs have emerged as a powerful tool for learning on complex non-euclidean data, which is becoming increasingly common for a variety of different applications. Yet, although their potential has been widely recognised in the machine learning community, graph learning is largely unexplored for downstream tasks such as robotics applications. To fully unlock their potential, hence, we propose a review of graph neural architectures from a robotics perspective. The paper covers the fundamentals of graph-based models, including their architecture, training procedures, and applications. It also discusses recent advancements and challenges that arise in applied settings, related for example to the integration of perception, decision-making, and control. Finally, the paper provides an extensive review of various robotic applications that benefit from learning on graph structures, such as bodies and contacts modelling, robotic manipulation, action recognition, fleet motion planning, and many more. This survey aims to provide readers with a thorough understanding of the capabilities and limitations of graph neural architectures in robotics, and to highlight potential avenues for future research.

show abstract

Section: Gnns For Multi-agent Systemsmentioning

confidence: 99%

Section: B Exploration and Navigationmentioning

confidence: 99%

Graph Learning in Robotics: A Survey

Pistilli,

Averta

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…There has been large success in generating high-performing cooperative teams using MARL in challenging problems such as game playing [4,22,20] and robotics [27]. Recently, the use of communication in MARL has become highly prevalent as agents can share important information to greatly improve team performance [6,26,15,10,39].…”

Section: Related Workmentioning

confidence: 99%

Heterogeneous Graph Attention Networks for Learning Diverse Communication

Seraj¹,

Wang²,

Paleja³

et al. 2021

Preprint

View full text Add to dashboard Cite

Multi-agent teaming achieves better performance when there is communication among participating agents allowing them to coordinate their actions for maximizing shared utility. However, when collaborating a team of agents with different action and observation spaces, information sharing is not straightforward and requires customized communication protocols, depending on sender and receiver types. Without properly modeling such heterogeneity in agents, communication becomes less helpful and could even deteriorate the multi-agent cooperation performance. We propose heterogeneous graph attention networks, called HetNet, to learn efficient and diverse communication models for coordinating heterogeneous agents towards accomplishing tasks that are of collaborative nature. We propose a Multi-Agent Heterogeneous Actor-Critic (MAHAC) learning paradigm to obtain collaborative per-class policies and effective communication protocols for composite robot teams. Our proposed framework is evaluated against multiple baselines in a complex environment in which agents of different types must communicate and cooperate to satisfy the objectives. Experimental results show that HetNet outperforms the baselines in learning sophisticated multi-agent communication protocols by achieving ∼10% improvements in performance metrics.

show abstract

“…Multi-agent reinforcement learning (MARL) is also an emerging method to solve the multi-agent problems that contain cooperation requirements such as simultaneous arrival and collision avoidance [9][10][11][12][13][14][15]. In [16], deep recurrent multi-agent actor-critic framework (R-MADDPG) is developed for cooperation under partial observable situations and limited communication capability such as network bandwidth.…”

Section: Introductionmentioning

confidence: 99%

Cooperative Planning for an Unmanned Combat Aerial Vehicle Fleet Using Reinforcement Learning

Yüksek

Demirezen²,

Tsourdos

et al. 2021

Journal of Aerospace Information Systems

View full text Add to dashboard Cite

In this study, reinforcement learning (RL)-based centralized path planning is performed for an unmanned combat aerial vehicle (UCAV) fleet in a human-made hostile environment. The proposed method provides a novel approach in which closing speed and approximate time-to-go terms are used in the reward function to obtain cooperative motion while ensuring no-fly-zones (NFZs) and time-of-arrival constraints. Proximal policy optimization (PPO) algorithm is used in the training phase of the RL agent. System performance is evaluated in two different cases. In case 1, the warfare environment contains only the target area, and simultaneous arrival is desired to obtain the saturated attack effect. In case 2, the warfare environment contains NFZs in addition to the target area and the standard saturated attack and collision avoidance requirements. Particle swarm optimization (PSO)-based cooperative path planning algorithm is implemented as the baseline method, and it is compared with the proposed algorithm in terms of execution time and developed performance metrics. Monte Carlo simulation studies are performed to evaluate the system performance. According to the simulation results, the proposed system is able to generate feasible flight paths in real-time while considering the physical and operational constraints such as acceleration limits, NFZ restrictions, simultaneous arrival, and collision avoidance requirements. In that respect, the approach provides a novel and computationally efficient method for solving the large-scale cooperative path planning for UCAV fleets.

show abstract

Scaling Up Multiagent Reinforcement Learning for Robotic Systems: Learn an Adaptive Sparse Communication Graph

Cited by 12 publications

References 24 publications

Graph Learning in Robotics: A Survey

Graph Learning in Robotics: A Survey

Heterogeneous Graph Attention Networks for Learning Diverse Communication

Cooperative Planning for an Unmanned Combat Aerial Vehicle Fleet Using Reinforcement Learning

Contact Info

Product

Resources

About