Ship detection and tracking is an important task in video surveillance in inland waterways. However, ships in inland navigation are faced with accidents such as collisions. For collision avoidance, we should strengthen the monitoring of navigation and the robustness of the entire system. Hence, this paper presents ship detection and tracking of ships using the improved You Only Look Once version 3 (YOLOv3) detection algorithm and Deep Simple Online and Real-time Tracking (Deep SORT) tracking algorithm. Three improvements are made to the YOLOv3 target detection algorithm. Firstly, the Kmeans clustering algorithm is used to optimize the initial value of the anchor frame to make it more suitable for ship application scenarios. Secondly, the output classifier is modified to a single Softmax classifier to suit our ship dataset which has three ship categories and mutual exclusion. Finally, Soft Non-Maximum Suppression (Soft-NMS) is introduced to solve the deficiencies of the Non-Maximum Suppression (NMS) algorithm when screening candidate frames. Results showed the mean Average Precision (mAP) and Frame Per Second (FPS) of the improved algorithm are increased by about 5% and 2, respectively, compared with the existing YOLOv3 detecting Algorithm. Then the improved YOLOv3 is applied in Deep Sort and the performance result of Deep Sort showed that, it has greater performance in complex scenes, and is robust to interference such as occlusion and camera movement, compared to state of art algorithms such as KCF, MIL, MOSSE, TLD, and Median Flow. With this improvement, it will help in the safety of inland navigation and protection from collisions and accidents.

In recent years, deep learning has been used in various applications including the classification of ship targets in inland waterways for enhancing intelligent transport systems. Various researchers introduced different classification algorithms, but they still face the problems of low accuracy and misclassification of other target objects. Hence, there is still a need to do more research on solving the above problems to prevent collisions in inland waterways. In this paper, we introduce a new convolutional neural network classification algorithm capable of classifying five classes of ships, including cargo, military, carrier, cruise and tanker ships, in inland waterways. The game of deep learning ship dataset, which is a public dataset originating from Kaggle, has been used for all experiments. Initially, the five pretrained models (which are AlexNet, VGG, Inception V3 ResNet and GoogleNet) were used on the dataset in order to select the best model based on its performance. Resnet-152 achieved the best model with an accuracy of 90.56%, and AlexNet achieved a lower accuracy of 63.42%. Furthermore, Resnet-152 was improved by adding a classification block which contained two fully connected layers, followed by ReLu for learning new characteristics of our training dataset and a dropout layer to resolve the problem of a diminishing gradient. For generalization, our proposed method was also tested on the MARVEL dataset, which consists of more than 10,000 images and 26 categories of ships. Furthermore, the proposed algorithm was compared with existing algorithms and obtained high performance compared with the others, with an accuracy of 95.8%, precision of 95.83%, recall of 95.80%, specificity of 95.07% and F1 score of 95.81%.