Research on intelligent collision avoidance decision-making of unmanned ship in unknown environments

Deep learning structure for collision avoidance planning of unmanned surface vessel

“…One more obstacle ship has been added in Figure 11.The initial positions of the own ship and other three obstacle ships are (6.6, 25), (26, 5.5), (7,6), (26,25), the initial heading is 120°, 320°, 210°, 40°and the initial velocity all set at 2 m/s, respectively. The inflection points occur on t 06 , t 07 and t 11 , and there is a turningback phenomenon in the trajectory figure (as depicted in Figure 11), which can explain the more complex situation of three obstacle ships all simultaneously reach the field energy threshold on Figure 12(b), so, the turning back is also a simplified choice when facing the complex situations under limited observation conditions for small type USV.…”

“…There exists the ability of learning knowledge by itself through continuous interaction with the external environment, which has the following two characteristics: one is to take the initiative to explore the behavior for the change of the environment, the second is to interact with the environment constantly, gain experience in accordance with the feedback signal, and acquire knowledge in the cycle of action evaluation, so as to improve the action strategy and finally adapt to the unknown environment, especially an unstructured and unpredictable environment. The deep Q-learning algorithm together with expert knowledge is adopted to develop a novel intelligent approach for automatic collision avoidance of multiple ships particularly in restricted waters, [23][24][25][26] Zhao studied the Q-network complying with the COLREGs to weaken the environmental disturbances, 27 but for meeting the requirements of ship kinematics and safety, the improved Q-learning method was proposed. 28 Given the historical information that the Q-learning algorithm performs in unknown environment, LSTM was selected as the neural network for generalization in collision avoidance strategy.…”

Real-Time Prediction of Large-Scale Ship Model Vertical Acceleration Based on Recurrent Neural Network

“…Beşikçi et al [13] developed a ship fuel consumption prediction system based on an ANN, which considers parameters such as ship speed, draught, propeller revolutions, and marine environment effects. Wang et al [14] established an intelligent collision avoidance model for unmanned ships through a deep reinforcement learning obstacle avoidance decision-making algorithm based on the Markov decision process. Ferrandis et al [15] compared the performance of a standard RNN with the gated recurrent units (GRU) and LSTM models by inputting random wave elevation under certain sea conditions and outputting the main motion of the ship, such as the pitch, heave, and roll.…”

Research on Real-Time Obstacle Avoidance Planning for an Unmanned Surface Vessel based on the Grid Cell Mechanism

“…Zhang et al (2014) proposed an adaptive risk aversion decision model using the Sarsa (state–action–reward–state–action) online strategy reinforcement learning algorithm, and combined this with a progressive corruption algorithm as a behaviour exploration strategy. Wang et al (2018) established a Markov decision-making method through reinforcement learning, and then the optimal strategy was solved by value function programming and simulated in static and dynamic obstacle environments. The deep reinforcement learning was also applied to train ships to avoid obstacles automatically (Shen et al, 2019).…”