Cross-Modality Person Re-Identification with Generative Adversarial Training

Abstract

Person re-identification (Re-ID) is an important task in video surveillance which automatically searches and identifies people across different cameras. Despite the extensive Re-ID progress in RGB cameras, few works have studied the Re-ID between infrared and RGB images, which is essentially a cross-modality problem and widely encountered in real-world scenarios. The key challenge lies in two folds, i.e., the lack of discriminative information to re-identify the same person between RGB and infrared modalities, and the difficulty to learn a robust metric towards such a large-scale cross-modality retrieval. In this paper, we tackle the above two challenges by proposing a novel cross-modality generative adversarial network (termed cmGAN). To handle the issue of insufficient discriminative information, we leverage the cutting-edge generative adversarial training to design our own discriminator to learn discriminative feature representation from different modalities. To handle the issue of large-scale cross-modality metric learning, we integrates both identification loss and cross-modality triplet loss, which minimize inter-class ambiguity while maximizing cross-modality similarity among instances. The entire cmGAN can be trained in an end-to-end manner by using standard deep neural network framework. We have quantized the performance of our work in the newly-released SYSU RGB-IR Re-ID benchmark, and have reported superior performance, i.e., Cumulative Match Characteristic curve (CMC) and Mean Average Precision (MAP), over the state-of-the-art works [Wu et al., 2017], respectively.

Co-Attentive Lifting for Infrared-Visible Person Re-Identification

“…We evaluate the proposed method (CoAL) on the SYSU-MM01 [50] and RegDB [32] datasets, and compare with state-of-the-art methods, including Zero-Pad [50], HCML [54], BDTR [55], cmGAN [8], MAC [53], D 2 RL [44], D-HSME [15], AlignGAN [41], MSR [12], CMSP [49], X-Modal [21], and Hi-CMD [7]. As it can be seen from the results presented in Table 1 and Table 2, our proposed method outperforms state-of-the-art methods significantly on both two datasets.…”

“…Wang et al [43] performed a comprehensive survey of heterogeneous person re-identification. Specifically, several types of cross-modality person ReID have been studied, including Image-to-Text cross-modality retrieval [24], Photo-to-Sketch cross-modality retrieval [34], and popular Infrared-to-Visible cross-modality retrieval [8,15,21,41,44,50,54,55,58]. Li et al [24] proposed that searching a person with free-form natural language descriptions can be widely applied in video surveillance and build a dataset for image-text cross-modality retrieval.…”

“…However, the LWL scheme can significantly increase the computational burden and memory consumption since several fully connections have to be added for the intermediate classification losses defined in Eqs. (6) and (8). Fully connection is computationally expensive, especially for the cases when the amount of training ID is large.…”

Enhancing the discriminative feature learning for visible-thermal cross-modality person re-identification

“…Adversarial Training [5]. Dai et al proposed a novel crossmodality generative adversarial network (termed cmGAN) to learn discriminative common representations.…”

Hetero-Center loss for cross-modality person Re-identification

“…Then, Ye et al [11] used triplet loss instead of contrastive loss to train an improved two-steam model named BDTR based on TONE, because contrastive loss is of weak flexibility in the feature embedding learning. Contemporarily, Dai et al [10] also adopted the joint supervision of triplet loss and CE loss to train a generative adversarial network named cmGAN which can learn modality-invariant feature representation.…”