This paper proposes an efficient method, based on reinforcement learning, to be used as ship controller in fast-time simulators within restricted channels. The controller must operate the rudder in a realistic manner in both time and angle variation so as to approximate human piloting. The method is well suited to scenarios where no previous navigation data is available; it takes into account, during training, both the effect of environmental conditions and also curves in channels. We resort to an asynchronous distributed version of the reinforcement learning algorithm Deep Q Network (DQN), handling channel segments as separate episodes and including curvature information as context variables (thus moving away from most work in the literature). We tested our proposal in the channel of Porto Sudeste, in the southern Brazilian coast, with realistic environment scenarios where wind and current incidence varies along the channel. The method keeps a simple representation and can be applied to any port channel configuration that respects local technical regulations.

Ship control in port channels is a challenging problem that has resisted automated solutions. In this paper we focus on reinforcement learning of control signals so as to steer ships in their maneuvers. The learning process uses fitted Q iteration together with a Ship Maneuvering Simulator. Domain knowledge is used to develop a compact state-space model; we show how this model and the learning process lead to ship maneuvering under difficult conditions.

No abstract

This paper proposes a machine learning agent for automatically navigating a vessel in a confined channel subject to environmental conditions. The agent is trained and tested using a Ship Maneuvering Simulator and is responsible for commanding the rudder, so as to keep the vessel inside the channel with minimum distance from the center line, and to reach the final part of the channel with a prescribed thruster rotation level. The algorithm is based on deep reinforcement learning method and uses an efficient state-space representation. The advantage of using reinforcement learning is that it does not require any expert to directly teach the agent how to behave under particular conditions. The novelty of this work is that: it does not require previous knowledge on the vessel dynamic model and the maneuvering scenario; it is robust against fluctuations of environmental forces such as wind and current; it considers discrete actions of rudder commands emulating the pilot actions in a real maneuver. The developed method is convenient for simulations in scenarios or areas that were never navigated before, in which no previous navigation data can be used to train a conventional supervised learning agent. One direct application for this work is the integration with a realistic fast-time maneuvering simulator for new ports or operations. Both training and validation experiments focused on the unsheltered approach channel of the Suape Port, in Brazil; these experiments were run in a SMH-USP maneuvering simulator (real environmental conditions measured on-site were employed in simulations).

Ship-to-ship (STS) operations have been widely applied for oil and cargo transfer in order to improve operational efficiency and reduce operational costs. Brazil has seen an increase in underway Ship-to-Ship operations due to the lack of available berths and terminals. The transfer operation is carried in sheltered locations in Brazil and abroad, but still has issues such as cable ruptures and high downtimes due to environmental conditions. This work proposes the development and validation of a control system for an innovative configuration for oil transfer between a DP shuttle and a conventional tanker, using a modified tandem configuration. The conventional vessel is underway and the DP vessel follows it with a lateral offset that provides a safe route for escape in case of any failure. The focus of the present paper is the control system design applied to the DP vessel that ensures the safe operation inside the offloading site. This control applies two individual sliding mode controllers for rudder and thruster control, with coefficients obtained from numerical simulations, associated with a line-of-sight strategy for course and speed over ground controllers. The set-point for the controller is obtained from the conventional vessel that is navigating ahead from the controlled vessel. Control performance and operation safety are evaluated through a set of real-time simulations of the transfer operation, where performance is evaluated through measurement of the Bow Loading System point position in both straight line navigation and in a 10km radius curve. The control robustness is evaluated by varying the environmental conditions in the simulations. Safety issues are also considered by evaluating the possibility of evasive maneuvers in case of component failure.