Research on Obstacle Avoidance Planning for UUV Based on A3C Algorithm

Abstract

Deep reinforcement learning is an artificial intelligence technology that combines deep learning and reinforcement learning and has been widely applied in multiple fields. As a type of deep reinforcement learning algorithm, the A3C (Asynchronous Advantage Actor-Critic) algorithm can effectively utilize computer resources and improve training efficiency by synchronously training Actor-Critic in multiple threads. Inspired by the excellent performance of the A3C algorithm, this paper uses the A3C algorithm to solve the UUV (Unmanned Underwater Vehicle) collision avoidance planning problem in unknown environments. This collision avoidance planning algorithm can have the ability to plan in real-time while ensuring a shorter path length, and the output action space can meet the kinematic constraints of UUVs. In response to the problem of UUV collision avoidance planning, this paper designs the state space, action space, and reward function. The simulation results show that the A3C collision avoidance planning algorithm can guide a UUV to avoid obstacles and reach the preset target point. The path planned by this algorithm meets the heading constraints of the UUV, and the planning time is short, which can meet the requirements of real-time planning.

A Comparative Analysis of Computational Intelligence Methods for Autonomous Navigation of Smart Ships

“…Analysis of the computational-intelligence-based collision avoidance methods for ships introduced in the recent literature allowed us to formulate the following summarizing remarks: The similar problems of collision avoidance and safe path planning also relate to other types of vehicles and moving objects, e.g., unmanned underwater vehicles (UUVs). Recent approaches for UUVs were proposed in [16], where a deep reinforcement learning algorithm was proposed for solving this task, and in [17], where the ant colony algorithm was applied. An algorithm based on reinforcement learning for unmanned aerial vehicles (UAVs) was proposed in [18].…”