Distributed Non-Communicating Multi-Robot Collision Avoidance via Map-Based Deep Reinforcement Learning

Abstract

It is challenging to avoid obstacles safely and efficiently for multiple robots of different shapes in distributed and communication-free scenarios, where robots do not communicate with each other and only sense other robots’ positions and obstacles around them. Most existing multi-robot collision avoidance systems either require communication between robots or require expensive movement data of other robots, like velocities, accelerations and paths. In this paper, we propose a map-based deep reinforcement learning approach for multi-robot collision avoidance in a distributed and communication-free environment. We use the egocentric local grid map of a robot to represent the environmental information around it including its shape and observable appearances of other robots and obstacles, which can be easily generated by using multiple sensors or sensor fusion. Then we apply the distributed proximal policy optimization (DPPO) algorithm to train a convolutional neural network that directly maps three frames of egocentric local grid maps and the robot’s relative local goal positions into low-level robot control commands. Compared to other methods, the map-based approach is more robust to noisy sensor data, does not require robots’ movement data and considers sizes and shapes of related robots, which make it to be more efficient and easier to be deployed to real robots. We first train the neural network in a specified simulator of multiple mobile robots using DPPO, where a multi-stage curriculum learning strategy for multiple scenarios is used to improve the performance. Then we deploy the trained model to real robots to perform collision avoidance in their navigation without tedious parameter tuning. We evaluate the approach with multiple scenarios both in the simulator and on four differential-drive mobile robots in the real world. Both qualitative and quantitative experiments show that our approach is efficient and outperforms existing DRL-based approaches in many indicators. We also conduct ablation studies showing the positive effects of using egocentric grid maps and multi-stage curriculum learning.

Robot Navigation in Crowd Based on Dual Social Attention Deep Reinforcement Learning

“…at is, the robot learns from the experience of pedestrian movement, understands the crowded scene, and encodes the human-computer interaction characteristics into the navigation strategy. e results in this field [12,20,21] have confirmed the superior performance of deep reinforcement learning in crowd perception navigation. ese deep reinforcement learning methods first collect relevant data from the surrounding people to build a value network, train in a reinforcement learning framework, and finally map the information to the control commands of the robot.…”

Deep reinforcement learning based mobile robot navigation: A review

“…(1) Expansion of network input Considering that the agent cannot uniquely distinguish its state based on current observations, the simplest solution is to add several previous observation frames as network inputs to improve its ability to distinguish among states [36,51,72,75,76] . In addition, previous rewards and actions also contain state information, so some studies have input previous rewards and actions to the network [33,44,63,77] .…”

“…This technique has been applied in DRL-based navigation task. Chen et al [76] introduced a two-stage training process for curriculum learning. In the first stage, they trained the policy in a random scenario with eight robots; and in the second stage, they trained the policy in both random and circular scenarios with 16 robots.…”

Non-Communication Decentralized Multi-Robot Collision Avoidance in Grid Map Workspace with Double Deep Q-Network

“…Researchers [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 ] have studied the decentralized multi-robot collision avoidance algorithm and some fruitful results have been achieved, such as collision avoidance with deep reinforcement learning (CADRL) [ 1 ], socially aware CADRL (SA-CADRL) [ 2 ], reciprocal velocity obstacle (RVO) [ 5 ]. The methods mentioned above were designed for cluttered workspaces.…”