Multi-Modal Contrastive Learning for LiDAR Point Cloud Rail-Obstacle Detection in Complex Weather

Abstract

Obstacle intrusion is a serious threat to the safety of railway traffic. LiDAR point cloud 3D semantic segmentation (3DSS) provides a new method for unmanned rail-obstacle detection. However, the inevitable degradation of model performance occurs in complex weather and hinders its practical application. In this paper, a multi-modal contrastive learning (CL) strategy, named DHT-CL, is proposed to improve point cloud 3DSS in complex weather for rail-obstacle detection. DHT-CL is a camera and LiDAR sensor fusion strategy specifically designed for complex weather and obstacle detection tasks, without the need for image input during the inference stage. We first demonstrate how the sensor fusion method is more robust under rainy and snowy conditions, and then we design a Dual-Helix Transformer (DHT) to extract deeper cross-modal information through a neighborhood attention mechanism. Then, an obstacle anomaly-aware cross-modal discrimination loss is constructed for collaborative optimization that adapts to the anomaly identification task. Experimental results on a complex weather railway dataset show that with an mIoU of 87.38%, the proposed DHT-CL strategy achieves better performance compared to other high-performance models from the autonomous driving dataset, SemanticKITTI. The qualitative results show that DHT-CL achieves higher accuracy in clear weather and reduces false alarms in rainy and snowy weather.

Robust Artificial Intelligence-Aided Multimodal Rail-Obstacle Detection Method by Rail Track Topology Reconstruction

“…We select two rail track instruction detection techniques-the 'LiDAR-based method [43]' and the 'fusion method [7]'-to compare with our approach. Inspired by [43][44][45][46], the 'LiDAR-based method' uses prior information to segment the rail track areas from the point cloud and calculates the intrusion. In Tables 2-4 of the updated manuscript, we selected the best results from [43,45,46] as the outcomes of the "LiDAR-based" technique and compared them with other methods.…”

“…Inspired by [43][44][45][46], the 'LiDAR-based method' uses prior information to segment the rail track areas from the point cloud and calculates the intrusion. In Tables 2-4 of the updated manuscript, we selected the best results from [43,45,46] as the outcomes of the "LiDAR-based" technique and compared them with other methods. However, we found that different methods led to some variations in results but there was no significant improvement or deterioration.…”