Cross-Modality Paired-Images Generation for RGB-Infrared Person Re-Identification

Abstract

RGB-Infrared (IR) person re-identification is very challenging due to the large cross-modality variations between RGB and IR images. The key solution is to learn aligned features to the bridge RGB and IR modalities. However, due to the lack of correspondence labels between every pair of RGB and IR images, most methods try to alleviate the variations with set-level alignment by reducing the distance between the entire RGB and IR sets. However, this set-level alignment may lead to misalignment of some instances, which limits the performance for RGB-IR Re-ID. Different from existing methods, in this paper, we propose to generate cross-modality paired-images and perform both global set-level and fine-grained instance-level alignments. Our proposed method enjoys several merits. First, our method can perform set-level alignment by disentangling modality-specific and modality-invariant features. Compared with conventional methods, ours can explicitly remove the modality-specific features and the modality variation can be better reduced. Second, given cross-modality unpaired-images of a person, our method can generate cross-modality paired images from exchanged images. With them, we can directly perform instance-level alignment by minimizing distances of every pair of images. Extensive experimental results on two standard benchmarks demonstrate that the proposed model favourably against state-of-the-art methods. Especially, on SYSU-MM01 dataset, our model can achieve a gain of 9.2% and 7.7% in terms of Rank-1 and mAP. Code is available at https://github.com/wangguanan/JSIA-ReID.

RGB-Infrared Person Re-Identification Via Multi-Modality Relation Aggregation and Graph Convolution Network

“…We compare our proposed method with other state-of-the-art methods, including Zero-padding, cmGAN [7], D2RL [12], JSIA [10], Hi-CMD [11], BDTR [25], X-modal [8], Align-GAN [9]. Result shown in Table 1 and 2 demonstrate that our proposed method outperforms the state-of-the-art method on both two datasets, we achieved 57.67% and 78.25% rank-1 accuracy on SYSU-MM01 and RegDB dataset, which surpassing previous state-of-the-art method by a large margin.…”

“…Also, difficulties like occlusions of human, low resolution, viewpoint change [5,6] faced in singlemodality person re-identification still exist in RGB-IR ReID problem. The key point in RGB-IR ReID task is reduce the modality discrepancy, existing methods [7,8,9,10,11,12] employ Generative Adversarial Networks (GAN) in RGB-IR Re-ID and generate fake infrared images to handle modality discrepancy problem, but the performance are not ideal because pictures in datasets are low resolution so generated pictures are blurred, and the structure information which is vital for Re-ID task in generated images are lost, generated images also brings some redundant information into datasets. In the other hand, some works [13,14,15,16,17] extract modality-specific and modality-shared features with independent feature extractor or branched network, and designed corresponding loss functions to narrow the distance between two modality feature distributions [13] or align features [9,17] to reduce modality discrepancy.…”

Towards a Unified Middle Modality Learning for Visible-Infrared Person Re-Identification

“…AlignGAN [34] is proposed to transform VIS images into IR images with effective constraints for both images and features alignment, and it achieves significant performance improvement. JSIA-ReID [35] is proposed to generate cross-modality paired-images and perform both global set-level and fine-grained instance-level alignments. Hi-CMD [2] introduces an effective generator to extract pose-and illumination-invariant features and maintain the identity characteristic of a specific person.…”

“…Although these methods improve the performance of VI-ReID to a certain extent, relying on the vanilla models does not take into account the non-linear relationship between the VIS and IR images. The other type of methods is the image-level methods (such as [34][35][36]), which aims to bridge the modality discrepancy by transforming the images from one modality to the other using the DNN-based image processing such as the generative adversarial networks (GANs) [11]. However, this strategy requires complex generative and discriminative networks and there is still a gap between the generated images and the real images.…”

Cross-Modality Person Re-Identification Based on Heterogeneous Center Loss and Non-Local Features

“…Cross-modality person re-identification [ 22 , 25 , 26 , 27 , 36 , 37 , 38 , 39 ] consists of solving the problem of person Re-ID under low light or night conditions. Wu et al [ 26 ] for the first time propose the deep zero-padding method for this task.…”