Unmanned Aerial Vehicle Path Planning Algorithm Based on Deep Reinforcement Learning in Large-Scale and Dynamic Environments

Abstract

Path planning is one of the key technologies for autonomous flight of Unmanned Aerial Vehicle. Traditional path planning algorithms have some limitations and deficiencies in the complex and dynamic environment. In this paper, we propose a deep reinforcement learning approach for threedimensional path planning by utilizing the local information and relative distance without global information.UAV can obtain the limited environmental information nearby in the actual scenario with limited sensor capabilities. Therefore, path planning can be formulated as a Partially Observable Markov Decision Process. The recurrent neural network with temporal memory is constructed to address the partial observability problem by extracting crucial information from historical state-action sequences. We develop an action selection strategy that combines the current reward value and the state-action value to reduce the meaningless exploration. In addition, we construct two sample memory pools and propose an adaptive experience replay mechanism based on the frequency of failure. The simulation experiment results show that our method has significant improvements over Deep Q-Network and Deep Recurrent Q-Network in terms of stability and learning efficiency. Our approach successfully plans a reasonable three-dimensional path in the large-scale and complex environment, and has the perfect ability to avoid obstacles.in the unknown environment.

Path Planning of Storage and Logistics Mobile Robot Based on ACA-E Algorithm

“…There are both black grids representing obstacles and white grids representing free walking in the grid, which makes the robot operating environment from complex to simple, and the path planning problem is relatively simple. Therefore, for the grid method, the most important thing is to determine the size of the divided grid, which will directly affect the operation of the algorithm and the final planning effect [19] (2) Geometric method uses geometric features (such as points, lines, and surfaces) to represent objects in the scene; abstracts the environmental information collected by the sensors carried by the robot into common geometric features, such as vertices, lines, curves, and corners; and then describes and records them with coordinates [20] (3) The expression of topological graph method is more abstract. It uses graphs to represent the spatial relationship between objects in the environment, and the nodes of the graph represent the feature points in the environment [21].…”

CLSQL: Improved Q-Learning Algorithm Based on Continuous Local Search Policy for Mobile Robot Path Planning

“…If the worst case happens, all the state–action pairs in the environment may be searched. Therefore, how to increase the search efficiency and convergence speed of the QL algorithm in path planning is a common challenge for scholars [ 18 , 19 , 20 ].…”

Adaptive UAV Swarm Mission Planning by Temporal Difference Learning

“…More recently, Xie et al [28] formulate UAV path planning as a POMDP. They use recurrent neurons to handle the partial observability by extracting crucial information from historical state-action pairs, and convolutional neurons to capture spatial feature information from the observation prior to determining the Q values of a state.…”