Progressively Hybrid Transformer for Multi-Modal Vehicle Re-Identification

Abstract

Multi-modal (i.e., visible, near-infrared, and thermal-infrared) vehicle re-identification has good potential to search vehicles of interest in low illumination. However, due to the fact that different modalities have varying imaging characteristics, a proper multi-modal complementary information fusion is crucial to multi-modal vehicle re-identification. For that, this paper proposes a progressively hybrid transformer (PHT). The PHT method consists of two aspects: random hybrid augmentation (RHA) and a feature hybrid mechanism (FHM). Regarding RHA, an image random cropper and a local region hybrider are designed. The image random cropper simultaneously crops multi-modal images of random positions, random numbers, random sizes, and random aspect ratios to generate local regions. The local region hybrider fuses the cropped regions to let regions of each modal bring local structural characteristics of all modalities, mitigating modal differences at the beginning of feature learning. Regarding the FHM, a modal-specific controller and a modal information embedding are designed to effectively fuse multi-modal information at the feature level. Experimental results show the proposed method wins the state-of-the-art method by a larger 2.7% mAP on RGBNT100 and a larger 6.6% mAP on RGBN300, demonstrating that the proposed method can learn multi-modal complementary information effectively.

GLFNet: Combining Global and Local Information in Vehicle Re-Recognition

“…In surveillance videos, vehicles, as crucial targets, have garnered widespread attention in computer vision, encompassing tasks such as recognition [ 4 , 5 ], detection [ 6 ], and classification [ 7 ]. The primary objective of vehicle re-identification (Re-ID) [ 8 , 9 , 10 , 11 , 12 ] is to accurately identify the same vehicle corresponding to a given detected vehicle in surveillance videos across different scenarios or time periods. Despite the adoption of deep learning networks by many researchers in recent years to extract vehicle features [ 13 , 14 ] and accomplish vehicle re-identification through feature matching, challenges persist in recognition accuracy due to variations in camera heights and angles.…”

Identity-Guided Spatial Attention for Vehicle Re-Identification

“…This paper investigates the challenge of identifying a specific vehicle from a vast image gallery database, known as vehicle re-identification [1][2][3][4][5]. The accuracy of this task relies heavily on the use of deep learning techniques [6][7][8][9] and computational resources, which are constrained by the availability of large-scale datasets.…”