COLREGs-Based Path Planning for USVs Using the Deep Reinforcement Learning Strategy

Wen, Naifeng; Long, Yundong; Zhang, Rubo; Liu, Guanqun; Wan, Wenjie; Jiao, Dian

doi:10.3390/jmse11122334

Cited by 3 publications

(1 citation statement)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This system, which is designed to adhere to the mandatory COLREGS results, is only tested in numerical simulations and so does not take into account the complexity of the real world and the highly dynamic and variable ocean environment. Another example of using RL to train agents to follow COLREGS is given in Wen et al (2023) . Here, a multi-agent team of ASVs are trained to avoid collisions with other agents in the system, environmental obstacles and other ships.…”

Section: Methodologies In Active Environmental Monitoringmentioning

confidence: 99%

A survey on autonomous environmental monitoring approaches: towards unifying active sensing and reinforcement learning

Mansfield,

Montazeri

2024

Front. Robot. AI

View full text Add to dashboard Cite

The environmental pollution caused by various sources has escalated the climate crisis making the need to establish reliable, intelligent, and persistent environmental monitoring solutions more crucial than ever. Mobile sensing systems are a popular platform due to their cost-effectiveness and adaptability. However, in practice, operation environments demand highly intelligent and robust systems that can cope with an environment’s changing dynamics. To achieve this reinforcement learning has become a popular tool as it facilitates the training of intelligent and robust sensing agents that can handle unknown and extreme conditions. In this paper, a framework that formulates active sensing as a reinforcement learning problem is proposed. This framework allows unification with multiple essential environmental monitoring tasks and algorithms such as coverage, patrolling, source seeking, exploration and search and rescue. The unified framework represents a step towards bridging the divide between theoretical advancements in reinforcement learning and real-world applications in environmental monitoring. A critical review of the literature in this field is carried out and it is found that despite the potential of reinforcement learning for environmental active sensing applications there is still a lack of practical implementation and most work remains in the simulation phase. It is also noted that despite the consensus that, multi-agent systems are crucial to fully realize the potential of active sensing there is a lack of research in this area.

show abstract

Section: Methodologies In Active Environmental Monitoringmentioning

confidence: 99%

A survey on autonomous environmental monitoring approaches: towards unifying active sensing and reinforcement learning

Mansfield,

Montazeri

2024

Front. Robot. AI

View full text Add to dashboard Cite

show abstract

An Improved NSGA-II Algorithm for MASS Autonomous Collision Avoidance under COLREGs

Liang,

Li,

Zhou

2024

JMSE

View full text Add to dashboard Cite

Autonomous collision avoidance decision making for maritime autonomous surface ships (MASS), as one of the key technologies for MASS autonomous navigation, is a research hotspot focused on by relevant scholars in the field of navigation. In order to guarantee the rationality, efficacy, and credibility of the MASS autonomous collision avoidance scheme, it is essential to design the MASS autonomous collision avoidance algorithm under the stipulations of the Convention on the International Regulations for Preventing Collisions at Sea (COLREGs). In order to enhance the autonomous collision avoidance decision-making capability of MASS in accordance with the relevant provisions of COLREGs, an improved NSGA-II autonomous collision avoidance decision-making algorithm based on the good point set method (GPS-NSGA-II) is proposed, which incorporates the collision hazard and the path cost of collision avoidance actions. The experimental results in the four simulation scenarios of head-on situation, overtaking situation, crossing situation, and multi-ship encounter situation demonstrate that the MASS autonomous collision avoidance decision making based on the GPS-NSGA-II algorithm under the constraints of COLREGs is capable of providing an effective collision avoidance scheme that meets the requirements of COLREGs in common encounter situations and multi-ship avoidance scenarios promptly, with a promising future application.

show abstract

Manipulation-Compliant Artificial Potential Field and Deep Q-Network: Large Ships Path Planning Based on Deep Reinforcement Learning and Artificial Potential Field

Xu,

Zhu,

Gao

et al. 2024

JMSE

View full text Add to dashboard Cite

Enhancing the path planning capabilities of ships is crucial for ensuring navigation safety, saving time, and reducing energy consumption in complex maritime environments. Traditional methods, reliant on static algorithms and singular models, are frequently limited by the physical constraints of ships, such as turning radius, and struggle to adapt to the maritime environment’s variability and emergencies. The development of reinforcement learning has introduced new methods and perspectives to path planning by addressing complex environments, achieving multi-objective optimization, and enhancing autonomous learning and adaptability, significantly improving the performance and application scope. In this study, we introduce a two-stage path planning approach for large ships named MAPF–DQN, combining Manipulation-Compliant Artificial Potential Field (MAPF) with Deep Q-Network (DQN). In the first stage, we improve the reward function in DQN by integrating the artificial potential field method and use a time-varying greedy algorithm to search for paths. In the second stage, we use the nonlinear Nomoto model for path smoothing to enhance maneuverability. To validate the performance and effectiveness of the algorithm, we conducted extensive experiments using the model of “Yupeng” ship. Case studies and experimental results demonstrate that the MAPF–DQN algorithm can find paths that closely match the actual trajectory under normal environmental conditions and U-shaped obstacles. In summary, the MAPF–DQN algorithm not only enhances the efficiency of path planning for large ships, but also finds relatively safe and maneuverable routes, which are of great significance for maritime activities.

show abstract

COLREGs-Based Path Planning for USVs Using the Deep Reinforcement Learning Strategy

Cited by 3 publications

References 34 publications

A survey on autonomous environmental monitoring approaches: towards unifying active sensing and reinforcement learning

A survey on autonomous environmental monitoring approaches: towards unifying active sensing and reinforcement learning

An Improved NSGA-II Algorithm for MASS Autonomous Collision Avoidance under COLREGs

Manipulation-Compliant Artificial Potential Field and Deep Q-Network: Large Ships Path Planning Based on Deep Reinforcement Learning and Artificial Potential Field

Contact Info

Product

Resources

About