Multisensor fusion‐based maritime ship object detection method for autonomous surface vehicles

Abstract

Autonomous surface vehicles face the challenge of accurately detecting nearby ships in the complex and ever‐changing maritime environment, which is vastly different from land areas. To address this issue, we propose an image‐based multisensor fusion object detection method that combines Light Detection and Rangings and cameras. Since point clouds have poor semantics, our method primarily relies on images, with point clouds used to support image detection. Our image detection scheme employs a tracking‐assisted detection method that leverages historical information to compensate for possible detection failures. Additionally, we designed a confidence‐association‐based fusion strategy to determine the final targets among the candidates. We conducted field experiments in an open‐sea area to demonstrate the accuracy and robustness of our method. The results of these experiments showed that our method is highly accurate and robust in challenging maritime scenarios. Our code and data set will be released on https://github.com/flakeice/mssd.

S2S-Sim: A Benchmark Dataset for Ship Cooperative 3D Object Detection

“…They constructed a point cloud dataset containing 7 categories of ships such as cargo ships and engineering ships. Zhang et al [15] collected a joint point cloud and image dataset containing 3180 frames in open waters, with 3 different scenes, for marine objectdetection tasks. Yao et al [17] equipped a yacht with a 16-line LiDAR, a monocular camera, an IMU, and a real-time kinematic (RTK) positioning module, and collected point cloud data in Xuanwu Lake, Nanjing, China, which can be used for multi-object tracking tasks.…”

“…Firstly, not all ships are equipped with AIS, making it difficult to identify ships or objects without AIS. Moreover, the AIS system refresh rate is not fixed and can be as long as 30 s, which is disadvantageous for real-time ship perception [15]. For radar, it is subject to high levels of noise, has a significant short-range blind zone, and detects a limited number of object points, making it unable to accurately discern object contours.…”

“…Conducting research on object perception in ship navigation scenarios also requires dataset construction first. Currently, several studies have constructed single-source ship navigation perception datasets for tasks such as object detection and tracking in navigation scenarios [15][16][17]. However, these datasets have not been widely promoted and applied.…”