Mapless Collaborative Navigation for a Multi-Robot System Based on the Deep Reinforcement Learning

Abstract

Compared with the single robot system, a multi-robot system has higher efficiency and fault tolerance. The multi-robot system has great potential in some application scenarios, such as the robot search, rescue and escort tasks, and so on. Deep reinforcement learning provides a potential framework for multi-robot formation and collaborative navigation. This paper mainly studies the collaborative formation and navigation of multi-robots by using the deep reinforcement learning algorithm. The proposed method improves the classical Deep Deterministic Policy Gradient (DDPG) to address the single robot mapless navigation task. We also extend the single-robot Deep Deterministic Policy Gradient algorithm to the multi-robot system, and obtain the Parallel Deep Deterministic Policy Gradient (PDDPG). By utilizing the 2D lidar sensor, the group of robots can accomplish the formation construction task and the collaborative formation navigation task. The experiment results in a Gazebo simulation platform illustrates that our method is capable of guiding mobile robots to construct the formation and keep the formation during group navigation, directly through raw lidar data inputs.

Deep reinforcement learning based mobile robot navigation: A review

“…In addition to avoiding collisions during navigation, multi-robots must often perform cooperative tasks such as maintaining formation. Chen et al [63] studied the problem of multi-robot formation. In their parallel DDPG method, multiple agents share experience memory data and navigation strategies.…”

“…(1) Expansion of network input Considering that the agent cannot uniquely distinguish its state based on current observations, the simplest solution is to add several previous observation frames as network inputs to improve its ability to distinguish among states [36,51,72,75,76] . In addition, previous rewards and actions also contain state information, so some studies have input previous rewards and actions to the network [33,44,63,77] . Another input expansion technique is the two-stream Q network proposed by Wang et al [78] , which adds the difference between two frames of laser scanning data.…”

“…In the first stage, they trained the policy in a random scenario with eight robots; and in the second stage, they trained the policy in both random and circular scenarios with 16 robots. Similar to curriculum learning [63] , multi-stage learning with policy evolution can also improve the DRL training efficiency under the sparse reward condition [40,86] .…”

Special Issue on Multi-Robot Systems: Challenges, Trends, and Applications

“…Finally, machine learning and especially reinforcement learning are increasingly popular for MRSs, especially for their operation in complex unstructured scenarios. Wenzhou Chen, Shizheng Zhou, Zaisheng Pan, Huixian Zheng, and Yong Liu [13] apply deep reinforcement learning for the collaborative formation and navigation of a robot fleet.…”

Autonomous Driving Control Using the DDPG and RDPG Algorithms

“…To our best knowledge, our work is the sole research work to consider both camera vision and distance information based on different methods simultaneously for a simulated vehicle model in order to present a comprehensive discussion. For example, the authors in [5] implemented their work with only a LiDAR sensor based on the DDPG method. In [6], although the authors realized their work based on LiDAR and an odometer, they only considered the -greedy policy of the DQN with different parameters to update the neural network.…”