Heave reduction of payload through crane control based on deep reinforcement learning using dual offshore cranes

Abstract

Offshore operation causes the dynamic motion of offshore cranes and payload by the ocean environment. The motion of the payload lowers the safety and efficiency of the work, which may increase the working time or cause accidents. Therefore, we design a control method for the crane using artificial intelligence (AI) to minimize the heave motion of the payload. Herein, reinforcement learning (RL), which calculates actions according to states, is applied. Furthermore, the deep deterministic policy gradient (DDPG) algorithm is used because the actions need to be determined in a continuous state. In the DDPG algorithm, the state is defined as the motion of the crane and speed of the wire rope, and the action is defined as the speed of the wire rope. In addition, the reward is calculated using the motion of the payload. In this study, the heave motion of the payload was reduced by developing an agent suitable for adjusting the length of the wire rope. The heave motion of the payload was compared in the non-learning condition of the RL-based and proportional integral differential (PID) controls, and an average reduction rate of 30% compared to PID control was confirmed. The RL-based control performed better than the PID control under learned conditions.