Attention guided cross-modal multispectral object detection

Abstract

The spatial offset between the visible and thermal images can pose challenges for accurately matching objects in both modalities, thus affecting the performance of object detection algorithms. To address this, we propose a feature alignment-based algorithm for unaligned RGB-T image object detection. Our approach admits the coarse-to-fine feature alignment strategy, incorporating an attention-guided feature offset prediction module. Additionally, a multi-headed selfattention mechanism is introduced to predict and correct the feature map offsets for visible images during the feature extraction stage. To further correct the offset between RGB-T features, a region of interest alignment module performs quadratic regression for each candidate frame in the pooling stage. Furthermore, our algorithm introduces a light-aware weighting module to adaptively adjust the contributions of different modalities by reweighting the region of interest features. Experimental results on the FLIRADAS dataset demonstrate that the proposed method achieves high accuracy and stability.