Research on Unmanned Surface Vehicles Environment Perception Based on the Fusion of Vision and Lidar

Abstract

The research of unmanned surface vehicles (usually called USV) is becoming a research hotspot, and real-time and accurate environmental perception is the key technology for achieving autonomous navigation and completing tasks of USV. In this paper, aiming at the technical problems of environmental perception of the USV, from the perspective of sensor multi-information fusion, we build a complete USV perception system (hardware platform and software system). This paper mainly studies objects on the water detection and obstacle avoidance methods based on the information fusion of lidar and vision. In addition, we also studies the method based on morphology to process the reflection of the objects on the water, and the sea-sky detection based on support vector machine (SVM) to assist the objects on the water detection. Through the construction of the USV environment perception system, improving the ability of USV environment perception, and ensuring the safety of autonomous navigation of USV. Our proposed methods have been tested by simulation experiments and actual marine experiment as well as the Hawaii Unmanned Surface Vehicles Challenge, which proves the practicability and stability of our method in the actual environment.

Multisensor fusion‐based maritime ship object detection method for autonomous surface vehicles

“…We followed the well‐developed calibration method in Zhang, Jiang et al (2021) to estimate the camera intrinsic parameters and camera‐LiDAR extrinsic parameters. The projected LiDAR points on the image using the estimated parameters are shown in Figure 7, and the estimated parameters are listed in Table 2.…”

“…Howerver, state-of-the-art (SOTA) performance is only evident when detecting large obstacles with salient features. Zhang, Jiang et al (2021) proposed a fusion method for detecting maritime objects that integrates information from both LiDARs and cameras.…”

“…Howerver, state‐of‐the‐art (SOTA) performance is only evident when detecting large obstacles with salient features. Zhang, Jiang et al (2021) proposed a fusion method for detecting maritime objects that integrates information from both LiDARs and cameras. Their method involves extracting targets from both cameras and LiDARs and using an intersection operation to combine the candidates from the images with those from point clouds.…”

A Surface Target Recognition Algorithm Based on Coordinate Attention and Double-Layer Cascade

“…The quality of images is more susceptible to the influence of weather conditions and background, resulting in low resolution, blurred edges, and easy confusion. Finally, surface target recognition technology is mainly used in the environment perception tasks that have more stringent requirements on the real time [6,7]. As shown in Figure 1, the three images indicated by the orange arrows are actually the same target, yet the sizes are completely different.…”

A Study Of Civil Engineering Education At Singapore Nanyang Technology University And At The University Of Florida

“…Environmental perception in the context of UVs typically refers to the ability of the vehicle to perceive and understand its surrounding environment in real-time [142]. This is a key technology for achieving autonomous navigation and completing tasks for UV swarms .…”