Learning to Cooperate in Decentralized Multi-robot Exploration of Dynamic Environments

Geng, Mingyang; Zhou, Xing; Ding, Bo; Wang, Huaimin; Zhang, Lei

doi:10.1007/978-3-030-04239-4_4

Cited by 15 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep Reinforcement Learning has been proved to be effective for enabling sophisticated and hard-to-design behaviors of robot individuals [22,23]. For the multi-robot exploration task, [24] proposes a learning-based method to enable the robots to actively learn the cooperation strategies as well as the action policies. Their method is robust enough to handle complex and dynamic environments and beats the performance of several "human-designed" methods.…”

Section: Learning-based Methods In Multi-robot Explorationmentioning

confidence: 99%

“…Their method is robust enough to handle complex and dynamic environments and beats the performance of several "human-designed" methods. The communication model used in [24] is CommNet [25], which simply averages the communication message to realize coordination. [26,27] improves the communication process by introducing the attention mechanism, which can precisely calculate whether the communication is necessary for each pair of agents in the exploration scenario.…”

Section: Learning-based Methods In Multi-robot Explorationmentioning

confidence: 99%

See 1 more Smart Citation

Efficient Multi-robot Exploration via Multi-head Attention-based Cooperation Strategy

Liu,

2019

Preprint

View full text Add to dashboard Cite

The goal of coordinated multi-robot exploration tasks is to employ a team of autonomous robots to explore an unknown environment as quickly as possible. Compared with "human-designed" methods, which began with heuristic and rule-based approaches, "learning-based" methods enable individual robots to learn sophisticated and hard-to-design cooperation strategies through deep reinforcement learning technologies. However, in decentralized multi-robot exploration tasks, learning-based algorithms are still far from being universally applicable to the continuous space due to the difficulties associated with area calculation and reward function designing; moreover, existing "learning-based" methods encounter problems when attempting to balance the "historical trajectory" issue and target area conflict problem. Furthermore, the scalability of these methods to a large number of agents is poor because of the exponential explosion problem of state space. Accordingly, this paper proposes a novel approach Multi-head Attention-based Multi-robot Exploration in Continuous Space (MAMECS) aimed at reducing the state space and automatically "learning" the cooperation strategies required for decentralized multi-robot exploration tasks in continuous space. Computational geometry knowledge is applied to describe the environment in continuous space and to design an improved reward function to ensure a superior exploration rate. Moreover, the multi-head attention mechanism employed helps to solve the "historical trajectory" issue in the decentralized multi-robot exploration task, as well as to reduce the quadratic increase of action space.

show abstract

Section: Learning-based Methods In Multi-robot Explorationmentioning

confidence: 99%

Section: Learning-based Methods In Multi-robot Explorationmentioning

confidence: 99%

Efficient Multi-robot Exploration via Multi-head Attention-based Cooperation Strategy

Liu,

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Consider the following robotic search and rescue scenario: A group of Unmanned Aerial Vehicles (UAVs) is sent to find the survivors in a group of high-rise buildings after an earthquake [ 1 ]. The harsh environmental conditions might cause individual robots to fail, or hackers might take control of some robots and force them to behave in misleading ways [ 2 ].…”

Section: Introductionmentioning

confidence: 99%

Attention-Based Fault-Tolerant Approach for Multi-Agent Reinforcement Learning Systems

Geng

Lan

2021

Entropy

Self Cite

View full text Add to dashboard Cite

The aim of multi-agent reinforcement learning systems is to provide interacting agents with the ability to collaboratively learn and adapt to the behavior of other agents. Typically, an agent receives its private observations providing a partial view of the true state of the environment. However, in realistic settings, the harsh environment might cause one or more agents to show arbitrarily faulty or malicious behavior, which may suffice to allow the current coordination mechanisms fail. In this paper, we study a practical scenario of multi-agent reinforcement learning systems considering the security issues in the presence of agents with arbitrarily faulty or malicious behavior. The previous state-of-the-art work that coped with extremely noisy environments was designed on the basis that the noise intensity in the environment was known in advance. However, when the noise intensity changes, the existing method has to adjust the configuration of the model to learn in new environments, which limits the practical applications. To overcome these difficulties, we present an Attention-based Fault-Tolerant (FT-Attn) model, which can select not only correct, but also relevant information for each agent at every time step in noisy environments. The multihead attention mechanism enables the agents to learn effective communication policies through experience concurrent with the action policies. Empirical results showed that FT-Attn beats previous state-of-the-art methods in some extremely noisy environments in both cooperative and competitive scenarios, much closer to the upper-bound performance. Furthermore, FT-Attn maintains a more general fault tolerance ability and does not rely on the prior knowledge about the noise intensity of the environment.

show abstract

“…Recently, deep learning techniques have been proven to be effective solutions that target learning control policies for robotics applications [ 5 , 6 , 7 , 8 ]. In particular, in the multi-agent area, deep reinforcement learning has shown its great potential for multiple agents to learn the cooperation strategies alongside their policy in specific tasks, which enables the sophisticated and hard-to-design behaviors of individual agents [ 9 ].…”

Section: Introductionmentioning

confidence: 99%

Learning to Cooperate via an Attention-Based Communication Neural Network in Decentralized Multi-Robot Exploration

Geng

Zhou

et al. 2019

Entropy

Self Cite

View full text Add to dashboard Cite

In a decentralized multi-robot exploration problem, the robots have to cooperate effectively to map a strange environment as soon as possible without a centralized controller. In the past few decades, a set of “human-designed” cooperation strategies have been proposed to address this problem, such as the well-known frontier-based approach. However, many real-world settings, especially the ones that are constantly changing, are too complex for humans to design efficient and decentralized strategies. This paper presents a novel approach, the Attention-based Communication neural network (CommAttn), to “learn” the cooperation strategies automatically in the decentralized multi-robot exploration problem. The communication neural network enables the robots to learn the cooperation strategies with explicit communication. Moreover, the attention mechanism we introduced additionally can precisely calculate whether the communication is necessary for each pair of agents by considering the relevance of each received message, which enables the robots to communicate only with the necessary partners. The empirical results on a simulated multi-robot disaster exploration scenario demonstrate that our proposal outperforms the traditional “human-designed” methods, as well as other competing “learning-based” methods in the exploration task.

show abstract

Learning to Cooperate in Decentralized Multi-robot Exploration of Dynamic Environments

Cited by 15 publications

References 11 publications

Efficient Multi-robot Exploration via Multi-head Attention-based Cooperation Strategy

Efficient Multi-robot Exploration via Multi-head Attention-based Cooperation Strategy

Attention-Based Fault-Tolerant Approach for Multi-Agent Reinforcement Learning Systems

Learning to Cooperate via an Attention-Based Communication Neural Network in Decentralized Multi-Robot Exploration

Contact Info

Product

Resources

About