Investigation and Imitation of Human Captains’ Maneuver Using Inverse Reinforcement Learning

Abstract

Automatic collision avoidance is of significant importance to prevent maritime collisions. Although many studies have been conducted in recent years, autonomous system has not completely replaced human captains since it is still difficult to imitate their complicated decisions. Thus, the present paper tries to investigate and imitate experienced captains’ maneuver using maximum entropy inverse reinforcement learning (MaxEnt IRL). We firstly verify that MaxEnt IRL can reproduce appropriate reward function from demonstrative trajectories. Afterwards, we conduct an experiment on a simulator where well-experienced captains maneuver in congested sea and estimate reward from the trajectories. Searching the route which maximizes the obtained reward, finally, we demonstrate the optimized route can avoid collision against multiple ships in compliance with the International Regulations for Preventing Collisions at Sea (COLREGs).

Devising an approach for the automated restoration of shipmaster’s navigational qualification parameters under risk conditions

“…During the analysis, when the speed of manifestation is more than two times, in difficult situations, the action was considered intuitive, in 56 % of cases it led to the application of the method of restoring the qualification parameters of operators-shipmasters. It is possible not to perceive such circumstances only in cases where experienced captains participate in the experiment, as discussed in study [35].…”