Obstacle detect based on improved neural network for fusion of radar and visual information

Abstract

Image target detection technology using deep learning has been fully developed in recent years, but in harsh and complex environments such as rain, snow, and darkness, the information collected by the image sensor will have a lot of noise, and it is easy to lose information about small distant targets and fuzzy targets, resulting in false detection and missed detection. Millimeter-wave(mmWave) radar uses electromagnetic waves for obstacle detection, and the detection performance in harsh environments basically does not degrade, in order to solve the situation that the detection accuracy is significantly reduced when single sensor environmental sensing technology is applied to complex scenes. In this paper, an enhanced neural network model for feature fusion is proposed by fusing radar and camera sensor features and combining them with an improved YOLO-S neural network. Firstly, the model is designed to convert the radar point cloud into image form while applying the BiFPN structure with integrated CA attention mechanism to the YOLOX-S neural network. By connecting radar features with image features and combining radar features with spatial attention weights, the useless information in the extracted features is suppressed while the key information is enhanced and the sensor features are complementary. The experimental results show that the fusion-enhanced network designed in this paper fully integrates the features of both sensors and provides better detection results and stronger robustness in detecting small distant objects and objects in dark and low-light environments.