Co-Attentive Lifting for Infrared-Visible Person Re-Identification

Abstract

Infrared-visible cross-modality person re-identification (IV-ReID) has attracted much attention with the popularity of dual-mode video surveillance systems, where the RGB mode works in the daytime and automatically switches to the infrared mode at night. Despite its significant application value, IV-ReID remains a difficult problem mainly due to two great challenges. First, it is difficult to identify persons in the infrared image, which lacks color and texture clues. Second, there is a significant gap between the infrared and visible modalities where appearances of the same person vary considerably. This paper proposes a novel attention-based approach to handle the two difficulties in a unified framework. 1) We propose an attention lifting mechanism to learn discriminative features in each modality. 2) We propose a co-attentive learning mechanism to bridge the gap between the two modalities. Our method only makes slight modifications of a given backbone network and requires small computation overhead while improving the performance significantly. We conduct extensive experiments to demonstrate the superiority of our proposed method. CCS CONCEPTS • Computing methodologies → Visual content-based indexing and retrieval; Appearance and texture representations; • Information systems → Nearest-neighbor search.

Feature Fusion and Center Aggregation for Visible-Infrared Person Re-Identification

“…In this subsection, we compare our proposed F 2 CALNet with the state-of-the-art (SOTA) visible-infrared person Re-ID approaches published in the last three years. These SOTA approaches contain Hi-CMD [48] adopting ID-discriminative factors to robust cross-modality match,JSIA [49], CoAL [54], and DF 2 AM [56]using two-level alignment approaches for cross-modality person matching task, HC [47] using multiple loss functions (enumerate angular triplet (EAT) loss, hetero-center loss, and cross-modality knowledge distillation (CMKD) loss) to enhance the feature distinctiveness, CMSP [51] and ATTRI [52] adopting extra constraints to increase intra-class cross-modality similarity while mitigating modality-specific information, HAT [53] generating additional modality between both visible and infrared modalities for alleviating the modality differences, NFS [55] utilizing a BN-oriented/feature search space to achieve standard optimization/automatic feature selection for the cross-modality work. MSO [67] proposed a perceptual edge features (PEF) loss to optimize their network.…”

Towards a Unified Middle Modality Learning for Visible-Infrared Person Re-Identification

“…We first compare the proposed MMN method with several other state-of-the-art methods to demonstrate the superiority of MMN method. The competing methods include the methods based on feature extraction (including Zero-Padding [40], HCML [44], MHM [41], BDTR [46], MAC [43], cmGAN [3], MSR [7], HSME [12], SNR [16], expAT [42], CMM [22], CMSP [39], SSFT [24], DDAA [45], CoAL [38], NFS [1]) and the methods based on image generation (including D 2 RL [36], JSIA-ReID [35], AlignGAN [34], Hi-CMD [2], DG-VAE [29], X-Modality [20]). The results on the RegDB and SYSU-MM01 datasets are reported in Table 1.…”

Visible-Infrared Person Re-Identification: A Comprehensive Survey and a New Setting

“…Ye et al [18] pointed out that the consistency at the feature and classifier levels is essential when dealing with modality differences. To learn discriminative representations in each modality, Wei et al [19] developed an attention-lifting mechanism. Wang et al [20] excavated spatial and channel information of images to reduce the discrepancy between two heterogeneous modalities.…”