Distributed and Scalable Cooperative Formation of Unmanned Ground Vehicles Using Deep Reinforcement Learning

Abstract

Cooperative formation control of unmanned ground vehicles (UGVs) has become one of the important research hotspots in the application of UGV and attracted more and more attention in the military and civil fields. Compared with traditional formation control algorithms, reinforcement-learning-based algorithms can provide a new solution with a lower complexity for real-time formation control by equipping UGVs with artificial intelligence. Therefore, in this paper, a distributed deep-reinforcement-learning-based cooperative formation control algorithm is proposed to solve the navigation, maintenance, and obstacle avoidance tasks of UGV formations. More importantly, the hierarchical triangular formation structure and the newly designed Markov decision process for UGV formations of leader and follower attributes make the control strategy learned by the algorithm reusable, so that the formation can arbitrarily increase the number of UGVs and realize a more flexible expansion. The effectiveness and scalability of the algorithm is verified by formation simulation experiments of different scales.

A Path-Planning Approach for an Unmanned Vehicle in an Off-Road Environment Based on an Improved A* Algorithm

“…In much off-road environmental work, unmanned vehicles play important roles, including rapid transportation, emergency rescue, and environmental reconnaissance [1][2][3]. The driving conditions of the unmanned vehicle in the off-road uncertain environment are different from those in the structured road condition, and there are many kinds of complex terrain in the off-road environment.…”