Automatic ship collision avoidance using deep reinforcement learning with LSTM in continuous action spaces

Sawada, Ryohei; Sato, Keiji; Majima, Takahiro

doi:10.1007/s00773-020-00755-0

Cited by 89 publications

(23 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To adds up the lengths of own and targeted is a simple way but not practical given the maneuverability of ships. Lyu [38] and Sawada [39] used the area with radius of 0.3 nm as the prohibited zone in their collision avoidance experiments, Sawada also took an effort with 0.5 nm but the performance was not as satisfied.…”

Section: Detection Rangesmentioning

confidence: 99%

A Coordination System between Decision Making and Controlling for Autonomous Collision Avoidance of Large Intelligent Ships

Zhou

Zhang

Wang

2021

JMSE

View full text Add to dashboard Cite

Large ships are typically with large inertia and longtime delay in motion, in prevailing collision avoidance methods, their maneuverability is generally neglected, there could be a dangerous situation if the system fails to control the ship course as ordered in a timely manner. This paper proposes a coordination system which consists of two algorithms for avoiding risk and then returning to scheduled waypoint. The avoiding risk algorithm are based on VO (velocity obstacle) method, the returning algorithm is derived from LOS (light of sight) guidance. For better performance, the ship model for simulation is a nonlinear Norrbin Model, with the controller improved by CGSA (closed loop gain shaping algorithm) method from traditional PID control, COLREGS (Convention on the International Regulations for Preventing Collisions at Sea) constrains are considered. To test the effectiveness of the proposed system, a series of complex scenarios including Imazu problem are applied.

show abstract

Section: Detection Rangesmentioning

confidence: 99%

A Coordination System between Decision Making and Controlling for Autonomous Collision Avoidance of Large Intelligent Ships

Zhou

Zhang

Wang

2021

JMSE

View full text Add to dashboard Cite

show abstract

“…A DRL-based COLREGs-compliant algorithm was proposed for multi-ship collision avoidance [ 29 ]. Sawada, et al [ 30 ] extended the DRL for continuous action spaces using an automatic collision avoidance algorithm. Researchers redesigned the long short-term memory (LSTM) network and trained the model in continuous action spaces.…”

Section: Related Workmentioning

confidence: 99%

A Real-Time Collision Avoidance Framework of MASS Based on B-Spline and Optimal Decoupling Control

Zhang

Wang

Chui

et al. 2021

Sensors

View full text Add to dashboard Cite

Real-time collision-avoidance navigation of autonomous ships is required by many application scenarios, such as carriage of goods by sea, search, and rescue. The collision avoidance algorithm is the core of autonomous navigation for Maritime autonomous surface ships (MASS). In order to realize real-time and free-collision under the condition of multi-ship encounter in an uncertain environment, a real-time collision avoidance framework is proposed using B-spline and optimal decoupling control. This framework takes advantage to handle the uncertain environment with limited sensing MASS which plans dynamically feasible, highly reliable, and safe feasible collision avoidance. First, owing to the collision risk assessment, a B-spline-based collision avoidance trajectory search (BCATS) algorithm is proposed to generate free-collision trajectories effectively. Second, a waypoint-based collision avoidance trajectory optimization is proposed with the path-speed decoupling control. Two benefits, a reduction of control cost and an improvement in the smoothness of the collision avoidance trajectory, are delivered. Finally, we conducted an experiment using the Electronic Chart System (ECS). The results reveal the robustness and real-time collision avoidance trajectory planned by the proposed collision avoidance system.

show abstract

“…Xie et al (2020) combined the long short-term memory neural network (LSTM) inverse model-based controller and the model-free A3C policy, to achieve ship collision avoidance under unknown environments. An automatic collision avoidance algorithm was proposed by combining the LSTM and RL in continuous action spaces (Sawada et al, 2021). However, deep learning has, upon the authors' best knowledge, yet been applied for end-to-end adaptive navigation, largely due to the difficulty by the complex and changeable marine environment.…”

Section: Introductionmentioning

confidence: 99%

Collision avoidance for autonomous ship using deep reinforcement learning and prior-knowledge-based approximate representation

et al. 2023

View full text Add to dashboard Cite

Reinforcement learning (RL) has shown superior performance in solving sequential decision problems. In recent years, RL is gradually being used to solve unmanned driving collision avoidance decision-making problems in complex scenarios. However, ships encounter many scenarios, and the differences in scenarios will seriously hinder the application of RL in collision avoidance at sea. Moreover, the iterative speed of trial-and-error learning for RL in multi-ship encounter scenarios is slow. To solve this problem, this study develops a novel intelligent collision avoidance algorithm based on approximate representation reinforcement learning (AR-RL) to realize the collision avoidance of maritime autonomous surface ships (MASS) in a continuous state space environment involving interactive learning capability like a crew in navigation situation. The new algorithm uses an approximate representation model to deal with the optimization of collision avoidance strategies in a dynamic target encounter situation. The model is combined with prior knowledge and International Regulations for Preventing Collisions at Sea (COLREGs) for optimal performance. This is followed by a design of an online solution to a value function approximation model based on gradient descent. This approach can solve the problem of large-scale collision avoidance policy learning in static-dynamic obstacles mixed environment. Finally, algorithm tests were constructed though two scenarios (i.e., the coastal static obstacle environment and the static-dynamic obstacles mixed environment) using Tianjin Port as an example and compared with multiple groups of algorithms. The results show that the algorithm can improve the large-scale learning efficiency of continuous state space of dynamic obstacle environment by approximate representation. At the same time, the MASS can efficiently and safely avoid obstacles enroute to reaching its target destination. It therefore makes significant contributions to ensuring safety at sea in a mixed traffic involving both manned and MASS in near future.

show abstract

Automatic ship collision avoidance using deep reinforcement learning with LSTM in continuous action spaces

Cited by 89 publications

References 17 publications

A Coordination System between Decision Making and Controlling for Autonomous Collision Avoidance of Large Intelligent Ships

A Coordination System between Decision Making and Controlling for Autonomous Collision Avoidance of Large Intelligent Ships

A Real-Time Collision Avoidance Framework of MASS Based on B-Spline and Optimal Decoupling Control

Collision avoidance for autonomous ship using deep reinforcement learning and prior-knowledge-based approximate representation

Contact Info

Product

Resources

About