Improving Description-Based Person Re-Identification by Multi-Granularity Image-Text Alignments

Abstract

Description-based person re-identification (Re-id) is an important task in video surveillance that requires discriminative cross-modal representations to distinguish different people. It is difficult to directly measure the similarity between images and descriptions due to the modality heterogeneity (the crossmodal problem). And all samples belonging to a single category (the fine-grained problem) makes this task even harder than the conventional image-description matching task. In this paper, we propose a Multi-granularity Image-text Alignments (MIA) model to alleviate the cross-modal fine-grained problem for better similarity evaluation in description-based person Re-id. Specifically, three different granularities, i.e., global-global, global-local and local-local alignments are carried out hierarchically. Firstly, the global-global alignment in the Global Contrast (GC) module is for matching the global contexts of images and descriptions. Secondly, the global-local alignment employs the potential relations between local components and global contexts to highlight the distinguishable components while eliminating the uninvolved ones adaptively in the Relation-guided Global-local Alignment (RGA) module. Thirdly, as for the local-local alignment, we match visual human parts with noun phrases in the Bi-directional Fine-grained Matching (BFM) module. The whole network combining multiple granularities can be end-to-end trained without complex preprocessing. To address the difficulties in training the combination of multiple granularities, an effective step training strategy is proposed to train these granularities step-by-step. Extensive experiments and analysis have shown that our method obtains the state-of-the-art performance on the CUHK-PEDES dataset and outperforms the previous methods by a significant margin.

Hierarchical Attention Image-Text Alignment Network For Person Re-Identification

“…We compare the results with feature embedding ( [4, 7, 28]) and attention based methods ( [10,13,29,30]) in Table 2. The results show that our HAITA-Net achieves better performance.…”

“…The results show that our HAITA-Net achieves better performance. Although MIA [29] and A-GANet [10] also learn the cross-modal representations by global or local associations, they weakly incorporate the importance of the informative word. Further, compared to A-GANet, we observe that for higher CMC ranks our model has significantly better performance.…”

Person Retrieval in Surveillance Using Textual Query: A Review

“…ResNet50 [127] LSTM [130] GLIA [82] CUHK-PEDES [85] Top-1 acc. ResNet-50 [127] Bi-GRU [145,146] MIA [135] CUHK-PEDES [85] Recall@1 (48.00%) Zhang et al [136] 2018…”

“…The GLIA approach gains a significant boost to the top-1 accuracy from the baselines and achieves 43. Multi-granularity Image-text Alignments (MIA) framework [135] adopts a multiple granularities (i.e., global-global, global-local, and local-local alignments) based approach for better similarity evaluations between text and image. The global context of image and description matches global-global granularity.…”

Person Re-Identification from different views based on dynamic linear combination of distances

“…Furthermore, Li et al [21] have adopted deep filter pairing neural network for person re-identification, and Wang et al [40] have investigated a joint learning framework for re-id while unifying Single-Image Representation (SIR) and Cross-Image Representation (CIR) via convolutional neural networks. More recently, a multi-granularity image-text alignment has been proposed for person re-identification [31].…”