Concise deep reinforcement learning obstacle avoidance for underactuated unmanned marine vessels

Cheng, Yin; Zhang, Wensheng

doi:10.1016/j.neucom.2017.06.066

Cited by 202 publications

(78 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As described in the article, the approach is model-free, requiring no prior knowledge of the system it is assigned to control. Another example of RL in USV steering is [3] who propose a deep RL approach for obstacle avoidance. As RL approaches, these require a pre-made reward function.…”

Section: Related Workmentioning

confidence: 99%

“…They are able to handle highly nonlinear relationships between their input and output [12,4], an ability which is necessary in order to perform end-to-end steering in USVs. Systems that couple deep ANNs with Reinforcement Learning (RL), have also proved to be able to learn complex tasks [14,11,10,8,3,7,15]. However, RL requires a known reward function to guide its training.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Generative Adversarial Immitation Learning for Steering an Unmanned Surface Vehicle

Vedeler¹,

Warakagoda

2020

nldl

View full text Add to dashboard Cite

The task of obstacle avoidance using maritime vessels, such as Unmanned Surface Vehicles (USV), has traditionally been solved using specialized modules that are designed and optimized separately. However, this approach requires a deep insight into the environment, the vessel, and their complex dynamics. We propose an alternative method using Imitation Learning (IL) through Deep Reinforcement Learning (RL) and Deep Inverse Reinforcement Learning (IRL) and present a system that learns an end-to-end steering model capable of mapping radar-like images directly to steering actions in an obstacle avoidance scenario. The USV used in the work is equipped with a Radar sensor and we studied the problem of generating a single action parameter, heading. We apply an IL algorithm known as generative adversarial imitation learning (GAIL) to develop an end-to-end steering model for a scenario where avoidance of an obstacle is the goal. The performance of the system was studied for different design choices and compared to that of a system that is based on pure RL. The IL system produces results that indicate it is able to grasp the concept of the task and that in many ways are on par with the RL system. We deem this to be promising for future use in tasks that are not as easily described by a reward function.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Generative Adversarial Immitation Learning for Steering an Unmanned Surface Vehicle

Vedeler¹,

Warakagoda

2020

nldl

View full text Add to dashboard Cite

show abstract

“…Considering ocean currents effect, a path planning algorithm based on RL is designed [237]. Aiming at obstacle avoidance, a concise deep reinforcement learning obstacle avoidance (CDRLOA) algorithm is proposed to overcome the usability issue caused by complicated control laws in traditional analysis methods [238].…”

Section: Artificial Intelligencementioning

confidence: 99%

State-of-the-Art Research on Motion Control of Maritime Autonomous Surface Ships

Wang

Liu

et al. 2019

JMSE

View full text Add to dashboard Cite

At present, with the development of waterborne transport vehicles, research on ship faces a new round of challenges in terms of intelligence and autonomy. The concept of maritime autonomous surface ships (MASS) has been put forward by the International Maritime Organization in 2017, in which MASS become the new focus of the waterborne transportation industry. This paper elaborates on the state-of-the-art research on motion control of MASS. Firstly, the characteristics and current research status of unmanned surface vessels in MASS and conventional ships are summarized, and the system composition of MASS is analyzed. In order to better realize the self-adaptability of the MASS motion control, the theory and algorithm of ship motion control-related systems are emphatically analyzed under the condition of classifying ship motion control. Especially, the application of intelligent algorithms in the ship control field is summarized and analyzed. Finally, this paper summarizes the challenges faced by MASS in the model establishment, motion control algorithms, and real ship experiments, and proposes the composition of MASS motion control system based on variable autonomous control strategy. Future researches on the accuracy and diversity of developments and applications to MASS motion control are suggested.

show abstract

“…Equation 4 allows us to obtain a smoother transition between risk-free states (i.e., known states) and risk states (i.e., unknown states). The parameter k has a double effect.…”

Section: The Continuous Risk Functionmentioning

confidence: 99%

“…There are many RL-based approaches to learn robotic tasks, but most of them do not face the problem of avoiding undesirable situations during training [4,13,17,29]. Commonly in these approaches, the agent receives a large negative reward for each visit to an undesirable or dangerous situation, and it must repeatedly experience these (possibly catastrophic) situations to learn how to avoid them.…”

Section: Introductionmentioning

confidence: 99%

Probabilistic Policy Reuse for Safe Reinforcement Learning

Garcı́a

Fernández

2018

ACM Trans. Auton. Adapt. Syst.

View full text Add to dashboard Cite

This work introduces Policy Reuse for Safe Reinforcement Learning (PR-SRL), an algorithm that combines Probabilistic Policy Reuse and teacher advice for safe exploration in dangerous and continuous state and action reinforcement learning problems in which the dynamic behavior is reasonably smooth, and the space is Euclidean. The algorithm uses a continuously increasing monotonic risk function which allows for the identification of the probability to end up in failure from a given state. Such a risk function is defined in terms of how far such a state is from the state space known by the learning agent. Probabilistic Policy Reuse is used to safely balance the exploitation of actual learned knowledge, the exploration of new actions, and the request of teacher advice in considered dangerous parts of the state space. Specifically, the π -reuse exploration strategy is used. Using experiments in the helicopter hover task and a business management problem, we show that the π -reuse exploration strategy can be used to completely avoid the visit to undesirable situations, while maintaining the performance (in terms of the classical long-term accumulated reward) of the final policy achieved.

show abstract

Concise deep reinforcement learning obstacle avoidance for underactuated unmanned marine vessels

Cited by 202 publications

References 15 publications

Generative Adversarial Immitation Learning for Steering an Unmanned Surface Vehicle

Generative Adversarial Immitation Learning for Steering an Unmanned Surface Vehicle

State-of-the-Art Research on Motion Control of Maritime Autonomous Surface Ships

Probabilistic Policy Reuse for Safe Reinforcement Learning

Contact Info

Product

Resources

About