Automatic Collision Avoidance Using Deep Reinforcement Learning with Grid Sensor

Sawada, Ryohei

doi:10.1007/978-3-030-37442-6_3

Cited by 5 publications

(8 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The policy and value function used in PPO are represented by deep neural networks. In the present study, we set a safe distance of 0.5 NM in the discrete action space in advance, but the trained model by previous approach [17] did not reach the sufficient performance. One of the possible reasons is that networks consist of only convolutional layers and fullconnected layers (FC) cannot store historical information of the environment.…”

Section: Structure Of Network and Update Methodsmentioning

confidence: 95%

“…The hyperparameters for PPO in continuous action spaces are provided in Table 3. The hyperparameters of the previous model in discrete action spaces are described in [17].…”

Section: Structure Of Network and Update Methodsmentioning

confidence: 99%

“…To process information of OZT, we use a virtual sensor called the grid sensor [17]. It is required to detect OZT and convert it into a form that can be easily used as an input of deep neural networks used in DRL.…”

Section: Detection Of Oztmentioning

confidence: 99%

“…They tested trained model for encounters with up to three target ships. One of the authors also proposed an automatic collision avoidance algorithm using proximal policy optimization (PPO) and a novel virtual sensor [17]. PPO is also one of the DRL algorithms and is used in not only playing video games but also controlling robots.…”

Section: Introductionmentioning

confidence: 99%

“…In the previous study [17], one of the authors used obstacle zone by target (OZT) [18] for collision risk assessment. OZT represents an area where a collision will happen in the future based on dynamic information of ships.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Automatic ship collision avoidance using deep reinforcement learning with LSTM in continuous action spaces

2020

Self Cite

View full text Add to dashboard Cite

This paper presents an automatic collision avoidance algorithm for ships using a deep reinforcement learning (DRL) in continuous action spaces. Obstacle zone by target (OZT) is used to compute an area where a collision will happen in the future based on dynamic information of ships. Agents of DRL detects the approach of multiple ships using a virtual sensor called the grid sensor. Agents learned collision avoidance maneuvering through Imazu problem, which is a scenario set of ship encounter situations. In this study, we propose a new approach for collision avoidance with a longer safe passing distance using DRL. We develop a novel method named inside OZT that expands OZT to improve the consistency of learning. We redesign the network using the long short-term memory (LSTM) cell and carried out training in continuous action spaces to train a model with longer safe distance than the previous study. The bow cross range in collision detection proposed in this paper is effective to COLREGs-compliant collision avoidance. The trained model has passed all scenarios of Imazu problem. The model is also validated by a test scenario which includes more ships than each scenario of Imazu problem.

show abstract

Section: Structure Of Network and Update Methodsmentioning

confidence: 95%

“…The hyperparameters for PPO in continuous action spaces are provided in Table 3. The hyperparameters of the previous model in discrete action spaces are described in [17].…”

Section: Structure Of Network and Update Methodsmentioning

confidence: 99%

Section: Detection Of Oztmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Automatic ship collision avoidance using deep reinforcement learning with LSTM in continuous action spaces

2020

Self Cite

View full text Add to dashboard Cite

show abstract

Framework of safety evaluation and scenarios for automatic collision avoidance algorithm

Sawada,

Sato,

Minami

2024

Ocean Engineering

View full text Add to dashboard Cite

Generalized Behavior Decision-Making Model for Ship Collision Avoidance via Reinforcement Learning Method

Guan

Zhao

Zhang

et al. 2023

JMSE

View full text Add to dashboard Cite

Due to the increasing number of transportation vessels, marine traffic has become more congested. According to the statistics, 89% to 95% of maritime accidents are related to human factors. In order to reduce marine incidents, ship automatic collision avoidance has become one of the most important research issues in the field of ocean engineering. A generalized behavior decision-making (GBDM) model, trained via a reinforcement learning (RL) algorithm, is proposed in this paper, and it can be used for ship autonomous driving in multi-ship encounter situations. Firstly, the obstacle zone by target (OZT) is used to calculate the area of future collisions based on the dynamic information of ships. Meanwhile, a virtual sensor called a grid sensor is taken as the input of the observation state. Then, International Regulations for Preventing Collision at Sea (COLREGs) is introduced into the reward function to make the decision-making fully comply with COLREGs. Different from the previous RL-based collision avoidance model, the interaction between the ship and the environment only works in the collision avoidance decision-making stage. Finally, 60 complex multi-ship encounter scenarios clustered by the COLREGs are taken as the ship’s GBDM model training environments. The simulation results show that the proposed GBDM model and training method has flexible scalability in solving the multi-ship collision avoidance problem complying with COLREGs in different scenarios.

show abstract

Automatic Collision Avoidance Using Deep Reinforcement Learning with Grid Sensor

Cited by 5 publications

References 8 publications

Automatic ship collision avoidance using deep reinforcement learning with LSTM in continuous action spaces

Automatic ship collision avoidance using deep reinforcement learning with LSTM in continuous action spaces

Framework of safety evaluation and scenarios for automatic collision avoidance algorithm

Generalized Behavior Decision-Making Model for Ship Collision Avoidance via Reinforcement Learning Method

Contact Info

Product

Resources

About