Finding the Shortest Course of a Ship Based on Reinforcement Learning Algorithm

Mitsubori, Kunihiko; Kamio, Takeshi; Tanaka, Takahiro

doi:10.9749/jin.110.9

Cited by 5 publications

(5 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Path planning applications train the agent so as to provide desired positions from the starting point to the goal, taking the ship to be a particle; the generated path is then tracked using any control strategy [17]. Given a sea area with current, [18] used Q-learning in its tabular form by discretizing the sea area uniformly and using abstract discrete actions. That idea was extended to environmental conditions by [19], and the approach was similarly implemented by [20], [21].…”

Section: B Related Workmentioning

confidence: 99%

Navigation in Restricted Channels Under Environmental Conditions: Fast-Time Simulation by Asynchronous Deep Reinforcement Learning

et al. 2020

View full text Add to dashboard Cite

This paper proposes an efficient method, based on reinforcement learning, to be used as ship controller in fast-time simulators within restricted channels. The controller must operate the rudder in a realistic manner in both time and angle variation so as to approximate human piloting. The method is well suited to scenarios where no previous navigation data is available; it takes into account, during training, both the effect of environmental conditions and also curves in channels. We resort to an asynchronous distributed version of the reinforcement learning algorithm Deep Q Network (DQN), handling channel segments as separate episodes and including curvature information as context variables (thus moving away from most work in the literature). We tested our proposal in the channel of Porto Sudeste, in the southern Brazilian coast, with realistic environment scenarios where wind and current incidence varies along the channel. The method keeps a simple representation and can be applied to any port channel configuration that respects local technical regulations.

show abstract

Section: B Related Workmentioning

confidence: 99%

Navigation in Restricted Channels Under Environmental Conditions: Fast-Time Simulation by Asynchronous Deep Reinforcement Learning

et al. 2020

View full text Add to dashboard Cite

show abstract

“…But, using our previous work [1], we can consider the tidal current effects. Os is the center in turning the ship's head and shows the ship's position (i.e., Os=(x, y)).…”

Section: Itmentioning

confidence: 99%

“…Vo is the velocity and its size is Vo. The dynamics is given by KT model [6] as follows: (1) where t5 is the rudder angle. T and K are the maneuvering performance parameters and they are given by K=Ko/(LslVo) and T= To(LslVo).…”

Section: Itmentioning

confidence: 99%

“…But, the optimality of the course and the interaction between maneuvering actions have not been sufficiently discussed yet. We regard multi-agent reinforcement learning system (MARLS) as a useful tool to brisk up these discussions, since ships have the special conditions in the maneuvering [1 ]- [4]. The conditions are as follows: 1) the dynamics is nonlinear, 2) there is no way to brake and go backward effectively, 3) the attitude is unstable at a low speed, and 4) the control tower with the strong authority does not exist.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-agent reinforcement learning system to find efficient courses for ships

Nakayama

Kamio

Mitsubori

et al. 2014

2014 IEEE 7th International Workshop on Computational Intelligence and Applications (IWCIA)

Self Cite

View full text Add to dashboard Cite

Although the ship transportation is important for low cost mass transit, the optimality of ships' courses and the interaction between maneuvering actions have not been sufficiently discussed yet. In order to brisk up these discussions, we have developed multi-agent reinforcement learning system (MARLS) to find ships' courses 111-141. Although our basic MARLS [3] can keep navigation rules [5], it may get inefficient courses including larger avoidance of collisions between ships. In this paper, we clarify the cause of this problem and propose a new MARLS controlled by the safety to overcome it. From numerical experiments, we have confirmed that our proposed MARLS can get more efficient courses than our basic MARLS.

show abstract

“…In recent years, since techniques of machine learning are developing rapidly, reinforcement learning, which is one of the machine learning, is begun to be applied to automatic collision avoidance. There are research which used Q-learning which is one of the reinforcement learning algorithms [13][14][15]. In the last few years, collision avoidance methods using DRL have also been proposed.…”

Section: Introductionmentioning

confidence: 99%

Automatic ship collision avoidance using deep reinforcement learning with LSTM in continuous action spaces

2020

View full text Add to dashboard Cite

This paper presents an automatic collision avoidance algorithm for ships using a deep reinforcement learning (DRL) in continuous action spaces. Obstacle zone by target (OZT) is used to compute an area where a collision will happen in the future based on dynamic information of ships. Agents of DRL detects the approach of multiple ships using a virtual sensor called the grid sensor. Agents learned collision avoidance maneuvering through Imazu problem, which is a scenario set of ship encounter situations. In this study, we propose a new approach for collision avoidance with a longer safe passing distance using DRL. We develop a novel method named inside OZT that expands OZT to improve the consistency of learning. We redesign the network using the long short-term memory (LSTM) cell and carried out training in continuous action spaces to train a model with longer safe distance than the previous study. The bow cross range in collision detection proposed in this paper is effective to COLREGs-compliant collision avoidance. The trained model has passed all scenarios of Imazu problem. The model is also validated by a test scenario which includes more ships than each scenario of Imazu problem.

show abstract

Finding the Shortest Course of a Ship Based on Reinforcement Learning Algorithm

Cited by 5 publications

References 8 publications

Navigation in Restricted Channels Under Environmental Conditions: Fast-Time Simulation by Asynchronous Deep Reinforcement Learning

Navigation in Restricted Channels Under Environmental Conditions: Fast-Time Simulation by Asynchronous Deep Reinforcement Learning

Multi-agent reinforcement learning system to find efficient courses for ships

Automatic ship collision avoidance using deep reinforcement learning with LSTM in continuous action spaces

Contact Info

Product

Resources

About