Navigation in Restricted Channels Under Environmental Conditions: Fast-Time Simulation by Asynchronous Deep Reinforcement Learning

Amendola, José; Miura, Lucas S.; Costa, Anna Helena Reali; Cozman, Fábio Gagliardi; Tannuri, Eduardo A.

doi:10.1109/access.2020.3015661

Cited by 16 publications

(5 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among these techniques, Deep learning-based RL architectures like deep Q-Network [24], deep deterministic policy gradient [25], PPO [26], and Soft Actor-Critic [27] have demonstrated exceptional capabilities in handling complex stochastic tasks. These methods have proven to be effective in areas such as cooperative path planning [28], navigating constrained environments [29], and patrolling water resources [30]. However, challenges arise in training tasks that require prolonged processes due to the inherent behavior of RL agents that rely on a specific, dense, and short-term reward system [31], [32].…”

Section: E Reinforcement Learningmentioning

confidence: 99%

USV Port Oil Spill Cleanup Using Hybrid Multi-Destination RL-CPP

Elmakis,

Degani

2023

IEEE Access

View full text Add to dashboard Cite

Human activities are the principal contributors to oil pollution in marine ecosystems, thereby causing severe ecological damage. The high volume of vessel traffic operating in these areas contributes to the rapid contamination of the marine ecosystem, leading to frequent oil spill events, particularly near ports where congestion is prevalent. Addressing this issue today necessitates the involvement of numerous skilled personnel committed to the task. This team undertakes the repetitive and tedious work of surveying the area, detecting spills, and employing various techniques to address each oil slick. The emergence of Unmanned Surface Vehicle (USV) technology has introduced a promising alternative capable of alleviating the process of continuous monitoring and cleaning operations in proximal shoreline areas. This paper addresses the problem of USV cleaning operations near the port. The proposed method synthesizes a hierarchical architecture that integrates traditional global path planning for multi-destination oil spills, along with coverage path planning based on reinforcement learning, to adapt to dynamically changing oil spills. This combined architecture results in a comprehensive solution, allowing navigation within the port's vicinity to address each occurrence of oil pollution. To evaluate the effectiveness of this approach, we conducted an elaborate simulation designed to replicate port activities. The findings of this paper indicate a significant reduction in pollution levels due to USV operation and underscore the ability to acquire complex policies for dynamic coverage planning through the use of a reinforcement learning framework.INDEX TERMS Autonomous agents, marine navigation, oil pollution, path planning, reinforcement learning.

show abstract

Section: E Reinforcement Learningmentioning

confidence: 99%

USV Port Oil Spill Cleanup Using Hybrid Multi-Destination RL-CPP

Elmakis,

Degani

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…These research also have only considered the constant radius of safety region to the obstacle. Trajectory planning on other kinds of restricted waters have been reported on: global trajectory planning and collision avoidance of an autonomous surface vessel at narrow ferry passage by safety region and collision region around moving and static obstacles with applying the hybrid dynamic window method (Serigstad et al, 2018); Navigation in restricted channels using reinforcement learning was done by Amendola et al (2020).…”

Section: Related Researchmentioning

confidence: 99%

Optimization on Planning of Trajectory and Control of Autonomous Berthing and Unberthing for the Realistic Port Geometry

Miyauchi,

Sawada,

Akimoto

et al. 2021

Preprint

View full text Add to dashboard Cite

To realize autonomous shipping, autonomous berthing and unberthing are some of the technical challenges. In the past, numerous research have been done on the optimization of trajectory planning of berthing problems. However, these studies assumed only a simple berth and did not consider obstacles. Optimization of trajectory planning on berthing and unberthing in actual ports must consider the spatial constraints and maintain sufficient distance to obstacles. The main contributions of this study are as follows: (i) a collision avoidance algorithm based on the ship domain which has variable size by the ship speed is proposed, to include the spatial constraints to optimization; (ii) the effect of wind disturbance is taken into account to the trajectory planning to make a feasible trajectory based on the capacity limit of actuators; (iii) showing that the optimization method for berthing is also eligible for the unberthing, which has been almost neglected; (iv) waypoints are included to the optimization process, to make optimization easier on practical applications. The authors tested the proposed method on two existing ports. The proposed method performed well on both the berthing and the unberthing problem and optimized the control input and the trajectory while avoiding collision with the complex obstacles.

show abstract

“…Besides those focused on berthing/unberthing maneuver, trajectory/path planning methods exist for generating reference trajectories for automation of ship operation. As an application of reinforcement learning, Amendola et al (2020) investigated the acquisition of control laws for narrow channel maneuver. Their method provided solutions that could be used as a trajectory plan.…”

Section: Introductionmentioning

confidence: 99%

Ship trajectory planning method for reproducing human operation at ports

Suyama,

Miyauchi,

Maki

2023

Preprint

View full text Add to dashboard Cite

Among ship maneuvers, berthing/unberthing maneuvers are one of the most challenging and stressful phases for captains. Concerning burden reduction on ship operators and preventing accidents, several researches have been conducted on trajectory planning to automate berthing/unberthing. However, few studies have aimed at assisting captains in berthing/unberthing. The trajectory to be presented to the captain should be a maneuver that reproduces human captain's control characteristics. The previously proposed methods cannot explicitly reflect the motion and navigation, which human captains pay particular attention to reduce the mental burden in the trajectory planning. Herein, mild constraints to the trajectory planning method are introduced. The constraints impose certain states (position, bow heading angle, ship speed, and yaw angular velocity), to be taken approximately at any given time. The introduction of this new constraint allows imposing careful trajectory planning (e.g., insitu turns at zero speed or a pause for safety before going astern), as if performed by a human during berthing/unberthing. The algorithm proposed herein was used to optimize the berthing/unberthing trajectories for a large car ferry. The results show that this method can generate the quantitatively equivalent trajectory recorded in the actual berthing/unberthing maneuver performed by a human captain.

show abstract

Navigation in Restricted Channels Under Environmental Conditions: Fast-Time Simulation by Asynchronous Deep Reinforcement Learning

Cited by 16 publications

References 18 publications

USV Port Oil Spill Cleanup Using Hybrid Multi-Destination RL-CPP

USV Port Oil Spill Cleanup Using Hybrid Multi-Destination RL-CPP

Optimization on Planning of Trajectory and Control of Autonomous Berthing and Unberthing for the Realistic Port Geometry

Ship trajectory planning method for reproducing human operation at ports

Contact Info

Product

Resources

About