Obtaining Human Experience for Intelligent Dredger Control: A Reinforcement Learning Approach

Abstract

This work presents a reinforcement learning approach for intelligent decision-making of a Cutter Suction Dredger (CSD), which is a special type of vessel for deepening harbors, constructing ports or navigational channels, and reclaiming landfills. Currently, CSDs are usually controlled by human operators, and the production rate is mainly determined by the so-called cutting process (i.e., cutting the underwater soil into fragments). Long-term manual operation is likely to cause driving fatigue, resulting in operational accidents and inefficiencies. To reduce the labor intensity of the operator, we seek an intelligent controller the can manipulate the cutting process to replace human operators. To this end, our proposed reinforcement learning approach consists of two parts. In the first part, we employ a neural network model to construct a virtual environment based on the historical dredging data. In the second part, we develop a reinforcement learning model that can lean the optimal control policy by interacting with the virtual environment to obtain human experience. The results show that the proposed learning approach can successfully imitate the dredging behavior of an experienced human operator. Moreover, the learning approach can outperform the operator in a way that can make quick responses to the change in uncertain environments.

Learning manipulation skills with demonstrations for the swing process control of dredgers

“…The work (Bai et al, 2019) investigates four machine learning algorithms to predict the productivity of a CSD, but it does not discuss how to control a CSD in an intelligent manner. Recently, RL has also been employed to study the swing process control problem in (Wei et al, 2019), but it only consider the simple case in discrete state and action spaces. Notably, such a method suffers from the curse of dimensionality.…”

“…erator, it is hard to accurately predict the changes of external environments so as to make quick and appropriate responses (Wei et al, 2019). Thus, it is intractable to construct a precise model that can handle all unforeseen situations in dredging various types of sand, clay or rock.…”

Productivity estimation of cutter suction dredger operation through data mining and learning from real-time big data

“…Li et al (2018) utilized actual monitoring data to recognize dredging cycles. Wei et al (2019a) developed a CSD intelligent decision system for control optimization using historical data and neural network learning methods. Wei et al (2019b) also proposed a method to improve the intelligent control of dredging vessel swing process using a dynamic and integrated SARSA-Lambda and linear neural network model.…”

Productivity regression analysis of cutter suction dredger considering operating characteristics and equipment status

“…10 In recent years, the development of artificial intelligence has brought opportunities for the prediction of dredger productivity. Wei et al 15 used reinforcement learning method to provide intelligent decision-making for cutter suction dredger. This method can make more beneficial decisions than manual based on artificial experience and considering a variety of constraints at the same time.…”