Reinforcement Learning-Based Autonomous Navigation and Obstacle Avoidance for USVs under Partially Observable Conditions

Yan, Nan; Huang, Shucai; Kong, Chao

doi:10.1155/2021/5519033

Cited by 16 publications

(4 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The method is tested for following a straight line and a sinusoid curve. Another study [21] discusses the wide usage of Unmanned Surface Vehicles (USV) in research, exploration, patrol, and defense. The study highlights autonomous navigation and obstacle avoidance as key technologies for successful USV missions.…”

Section: Related Workmentioning

confidence: 99%

Uncertainity and Noise Aware Decision Making for Autonomous Vehicles -A Bayesian Approach

Sachdeva,

Gakhar,

Awasthi

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

Uncertainity and Noise Aware Decision Making for Autonomous Vehicles -A Bayesian Approach

Sachdeva,

Gakhar,

Awasthi

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Accordingly, autonomous ships learned human-defined rules in those environments [25]. The RL algorithm was also applied to decision-making on autonomous navigation under partial observation considering a real environment [50]. The multi-agent algorithm was applied to maintain the formation during navigation and path finding, in which several autonomous ships sailed [51].…”

Section: Rl For a Hierarchical Autonomous Ship Taskmentioning

confidence: 99%

Continuous Autonomous Ship Learning Framework for Human Policies on Simulation

2022

View full text Add to dashboard Cite

Considering autonomous navigation in busy marine traffic environments (including harbors and coasts), major study issues to be solved for autonomous ships are avoidance of static and dynamic obstacles, surface vehicle control in consideration of the environment, and compliance with human-defined navigation rules. The reinforcement learning (RL) algorithm, which demonstrates high potential in autonomous cars, has been presented as an alternative to mathematical algorithms and has advanced in studies on autonomous ships. However, the RL algorithm, through interactions with the environment, receives relatively fewer data from the marine environment. Moreover, the open marine environment causes difficulties for autonomous ships in learning human-defined navigation rules because of excessive degrees of freedom. This study proposes a sustainable, intelligent learning framework for autonomous ships (ILFAS), which helps solve these difficulties and learns navigation rules specified by human beings through neighboring ships. The application of case-based RL enables the participation of humans in the RL learning process through neighboring ships and the learning of human-defined rules. Cases built as curriculums can achieve high learning effects with fewer data along with the RL of layered autonomous ships. The experiment aims at autonomous navigation from a harbor, where marine traffic occurs on a neighboring coast. The learning results using ILFAS and those in an environment where random marine traffic occurs are compared. Based on the experiment, the learning time was reduced by a tenth. Moreover, the success rate of arrival at a destination was higher with fewer controls than the random method in the new marine traffic scenario. ILFAS can continuously respond to advances in ship manufacturing technology and changes in the marine environment.

show abstract

“…However, these works use traditional 2D simulation-based conditions that neglect the complex nature of the real world. While more recent works have incorporated 3D simulations in their experiments [19], they do not present an end-toend AMSV system. In addition, most of these works use prerecorded data, which limits their accuracy and efficacy.…”

Section: Related Workmentioning

confidence: 99%

Towards Integrated Digital-twins: An Application Framework for Autonomous Maritime Surface Vessel Development

Raza¹,

Prokopova²,

Huseynzade³

et al. 2022

Preprint

View full text Add to dashboard Cite

The use of digital twins for the development of Autonomous Maritime Surface Vessels (AMSVs) has enormous potential to resolve the increasing need for water-based navigation and safety at the seas. Aiming at the problem of lack of broad and integrated digital twin implementations with live data along with the absence of a digital twin-driven framework for AMSV design and development, an application framework for the development of a fully autonomous vessel using an integrated digital twin in a 3D simulation environment has been presented. Our framework has four layers to ensure that the simulation and the real-world boat as well as the environment are as close as possible. Åboat, an experimental research platform for maritime automation and autonomous surface ship applications, equipped with two trolling electric motors, cameras, LiDARs, IMU and GPS has been used as the case study to provide a proof of concept. Åboat and its sensors, alongwith the environment have been replicated in a 3D simulation environment. Using the proposed application framework, we develop obstacle detection and path planning systems based on machine learning which leverage live data from a 3D simulation environment to mirror the complex dynamics of the real world.

show abstract

Reinforcement Learning-Based Autonomous Navigation and Obstacle Avoidance for USVs under Partially Observable Conditions

Cited by 16 publications

References 34 publications

Uncertainity and Noise Aware Decision Making for Autonomous Vehicles -A Bayesian Approach

Uncertainity and Noise Aware Decision Making for Autonomous Vehicles -A Bayesian Approach

Continuous Autonomous Ship Learning Framework for Human Policies on Simulation

Towards Integrated Digital-twins: An Application Framework for Autonomous Maritime Surface Vessel Development

Contact Info

Product

Resources

About