Research on Method of Collision Avoidance Planning for UUV Based on Deep Reinforcement Learning

Abstract

A UUV can perform tasks such as underwater surveillance, reconnaissance, surveillance, and tracking by being equipped with sensors and different task modules. Due to the complex underwater environment, the UUV must have good collision avoidance planning algorithms to avoid various underwater obstacles when performing tasks. The existing path planning algorithms take a long time to plan and have poor adaptability to the environment. Some collision-avoidance planning algorithms do not take into account the kinematic limitations of the UUV, thus placing high demands on the performance and control algorithms of UUV. This article proposes a PPO−DWA collision avoidance planning algorithm for the UUV under static unknown obstacles, which is based on the proximal policy optimization (PPO) algorithm and the dynamic window approach (DWA). This algorithm acquires the obstacle information from forward-looking sonar as input and outputs the corresponding continuous actions. The PPO−DWA collision avoidance planning algorithm consists of the PPO algorithm and the modified DWA. The PPO collision avoidance planning algorithm is only responsible for outputting the continuous angular velocity, aiming to reduce the difficulty of training neural networks. The modified DWA acquires obstacle information and the optimal angular velocity from the PPO algorithm as input, and outputs of the linear velocity. The collision avoidance actions output by this algorithm meet the kinematic constraints of UUV, and the algorithm execution time is relatively short. The experimental data demonstrates that the PPO−DWA algorithm can effectively plan smooth collision-free paths in complex obstacle environments, and the execution time of the algorithm is acceptable.

An Obstacle Avoidance Strategy for AUV Based on State-Tracking Collision Detection and Improved Artificial Potential Field

“…However, AUVs still require an obstacle avoidance algorithm to navigate around potential obstacles along the planned trajectory. Several solutions are available for AUV DOA, including rapidly exploring random trees (RRTs) [1], fuzzy logic [2], the neural network (NN), reinforcement learning (RL), deep reinforcement learning (DRL) [3], and the artificial potential field (APF) [4][5][6]. The RRT algorithm exhibits a robust capability for detecting unknown obstacles and is well-suited for addressing obstacle avoidance issues in high-dimensional environments; however, its real-time performance is relatively poor [7,8].…”