HSME: Hypersphere Manifold Embedding for Visible Thermal Person Re-Identification

Abstract

Person Re-identification(re-ID) has great potential to contribute to video surveillance that automatically searches and identifies people across different cameras. Heterogeneous person re-identification between thermal(infrared) and visible images is essentially a cross-modality problem and important for night-time surveillance application. Current methods usually train a model by combining classification and metric learning algorithms to obtain discriminative and robust feature representations. However, the combined loss function ignored the correlation between classification subspace and feature embedding subspace. In this paper, we use Sphere Softmax to learn a hypersphere manifold embedding and constrain the intra-modality variations and cross-modality variations on this hypersphere. We propose an end-to-end dualstream hypersphere manifold embedding network(HSMEnet) with both classification and identification constraint. Meanwhile, we design a two-stage training scheme to acquire decorrelated features, we refer the HSME with decorrelation as D-HSME. We conduct experiments on two crossmodality person re-identification datasets. Experimental results demonstrate that our method outperforms the state-of-the-art methods on two datasets. On RegDB dataset, rank-1 accuracy is improved from 33.47% to 50.85%, and mAP is improved from 31.83% to 47.00%.

HPILN: a feature learning framework for cross‐modality person re‐identification

“…For performance measure, the rank‐1, ‐10, ‐20 accuracies of CMC and mAP are used to show the clear performance superiority of our HPILN method. The comparison takes advantage of seven state‐of‐the‐art methods: zero‐padding [7], cmGAN [9], bi‐directional dual‐constrained top‐ranking (BDTR) [8], inter‐channel pair between the visible‐light and thermal images + multi‐scale Retinex (IPVT‐1 + MSR) [10], D 2 RL [11], bi‐directional center‐constrained top‐ranking (eBDTR) [27] and D‐hypersphere manifold embedding (HSME) [28].…”

“…In addition, other existing methods are used for comparison, including handcrafted features such as Histograms of Oriented Gradient (HOG) [29] and Local Maximal Occurrence (LOMO) [30], cross‐domain models such as Common Discriminant Feature Extraction (CDFE) [31] and Camera coRrelation Aware Feature augmenTation (CRAFT) [32], canonical correlation analysis (CCA) [33], one‐stream and two‐stream networks [7], and metric learning method Local Fisher Discriminant Analysis (LFDA) [34]. Most of the results were obtained from the references [7–11, 27, 28].…”

Co-Attentive Lifting for Infrared-Visible Person Re-Identification

“…We evaluate the proposed method (CoAL) on the SYSU-MM01 [50] and RegDB [32] datasets, and compare with state-of-the-art methods, including Zero-Pad [50], HCML [54], BDTR [55], cmGAN [8], MAC [53], D 2 RL [44], D-HSME [15], AlignGAN [41], MSR [12], CMSP [49], X-Modal [21], and Hi-CMD [7]. As it can be seen from the results presented in Table 1 and Table 2, our proposed method outperforms state-of-the-art methods significantly on both two datasets.…”

“…Wang et al [43] performed a comprehensive survey of heterogeneous person re-identification. Specifically, several types of cross-modality person ReID have been studied, including Image-to-Text cross-modality retrieval [24], Photo-to-Sketch cross-modality retrieval [34], and popular Infrared-to-Visible cross-modality retrieval [8,15,21,41,44,50,54,55,58]. Li et al [24] proposed that searching a person with free-form natural language descriptions can be widely applied in video surveillance and build a dataset for image-text cross-modality retrieval.…”

“…Ye et al [55] proposed a dual-constrained pair-wise triplet loss to directly optimize cross-modal matching. Hao et al [15] proposed a unified cross-modal feature learning and metric learning approach in a hyper-sphere manifold space. Recently, a few of methods based on generative adversarial networks (GANs) [13] have been proposed to narrow the semantic gap between the infrared and visible modalities.…”