Sea–Sky Line and Its Nearby Ships Detection Based on the Motion Attitude of Visible Light Sensors

Abstract

In the maritime scene, visible light sensors installed on ships have difficulty accurately detecting the sea–sky line (SSL) and its nearby ships due to complex environments and six-degrees-of-freedom movement. Aimed at this problem, this paper combines the camera and inertial sensor data, and proposes a novel maritime target detection algorithm based on camera motion attitude. The algorithm mainly includes three steps, namely, SSL estimation, SSL detection, and target saliency detection. Firstly, we constructed the camera motion attitude model by analyzing the camera's six-degrees-of-freedom motion at sea, estimated the candidate region (CR) of the SSL, then applied the improved edge detection algorithm and the straight-line fitting algorithm to extract the optimal SSL in the CR. Finally, in the region of ship detection (ROSD), an improved visual saliency detection algorithm was applied to extract the target ships. In the experiment, we constructed SSL and its nearby ship detection dataset that matches the camera’s motion attitude data by real ship shooting, and verified the effectiveness of each model in the algorithm through comparative experiments. Experimental results show that compared with the other maritime target detection algorithm, the proposed algorithm achieves a higher detection accuracy in the detection of the SSL and its nearby ships, and provides reliable technical support for the visual development of unmanned ships.

A Horizon Line Annotation Software for Streamlining Autonomous Sea Navigation Experiments

“…Third, many works used the horizon to segment the image (into sea and non-sea region). In the same context, the horizon feature provides computational benefits to the object detection task by reducing the search region down to that around the horizon or by eliminating the sky pixels that will obviously not depict any obstacle [5,[18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35]. Fourth, even after the detection of maritime objects on the image coordinates, some researchers used the horizon line to geo-localize surrounding objects and obstacles [36][37][38][39], i.e., find the distance in meters from the camera to the obstacle.…”

A Review of Methods for Ship Detection with Electro-Optical Images in Marine Environments

“…Visible Chen [12], Cane [13], Marie [14], Chen [15], Chen [16], Liu [17], Shan [18], Gal [19], Lin [20], Chen [21], Lee [22], Feng [23], Shan [24], Fefilatyev [25] IR N/A Tang [26], Liu [27], Hu [28], Lin [29] MWIR Özertem [30], Wang [31] LWIR Sun [32], Lu [33], Bai [34], Leira [35], Bai [36], Mumtaz [37], Singh [38], Zhang [39], Xu [40], Zhou [41], Schöller [42], Li [43], Westlake [44] Visible + IR N/A Islam [45], Wei [46] Visible + MWIR Nita [47] Visible + LWIR Zhang [48], Ribeiro [49], Farahnakian [50] Visible + SWIR + LWIR Stets [51] Visible + SWIR + MWIR + LWIR Bouma [52] Visible: refers to visible-band images; IR: infrared images, including short-wave infrared (SWIR), medium-wave infrared (MWIR), and long-wave infrared (LWIR); N/A, means that the type of image was not specified in the paper. from the statistical data that infrared images are used the most, followed by visible images.…”

“…For example, Tang [26] used gradient and shape information, Lu [33] used the distance rule, and Chen [12] used the mutation of gray values. Some researchers also use a fixed-height search range near the SSL (Fefilatyev [25]) or cut the image directly near the SSL to extract ROIs (Shan [18]).…”

Remote Sensing in Vessel Detection and Navigation

“…Shan et al [ 9 ] address the problem of accurate detection of the coastline and nearby ships with the available sensors due to the complex environment and six degrees of freedom of movement. The article combines data from cameras and inertial sensors and proposes an innovative algorithm for detecting sea targets based on camera position in relation to movement.…”