Long-Term Person Re-Identification Based on Appearance and Gait Feature Fusion under Covariate Changes

Abstract

Person re-identification(Re-ID) technology has been a research hotspot in intelligent video surveillance, which accurately retrieves specific pedestrians from massive video data. Most research focuses on the short-term scenarios of person Re-ID to deal with general problems, such as occlusion, illumination change, and view variance. The appearance change or similar appearance problem in the long-term scenarios has has not been the focus of past research. This paper proposes a novel Re-ID framework consisting of a two-branch model to fuse the appearance and gait feature to overcome covariate changes. Firstly, we extract the appearance features from a video sequence by ResNet50 and leverage average pooling to aggregate the features. Secondly, we design an improved gait representation to obtain a person’s motion information and exclude the effects of external covariates. Specifically, we accumulate the difference between silhouettes to form an active energy image (AEI) and then mask the mid-body part in the image with the Improved-Sobel-Masking operator to extract the final gait representation called ISMAEI. Thirdly, we combine appearance features with gait features to generate discriminative and robust fused features. Finally, the Euclidean norm is adopted to calculate the distance between probe and gallery samples for person Re-ID. The proposed method is evaluated on the CASIA Gait Database B and TUM-GAID datasets. Compared with state-of-the-art methods, experimental results demonstrate that it can perform better in both Rank-1 and mAP.

UnA-Mix: Rethinking Image Mixtures for Unsupervised Person Re-Identification

“…The input data are mixed with the corresponding labels in a seemingly meaningless way, which can effectively guide the training of the network model. This has been validated in supervised learning, as shown in Equations ( 1)- (3).…”

“…Person re-identification (Re-ID) is defined as the technique of determining the presence of a specific person in an image or video sequence by using computer vision techniques, and it is widely regarded as a sub-problem of image retrieval. It can compensate for the visual limitations of fixed ultra-long-range visual sensors [1][2][3]. Because of its importance in long-distance intelligent surveillance systems, supervised person Re-ID has been widely deployed in real-world scenarios.…”

An end-to-end gait recognition system for covariate conditions using custom kernel CNN

“…A novel framework for person re-identification is proposed, aiming to overcome covariate changes by fusing appearance and gait features [ 37 ]. The framework consists of a two-branch model.…”

A comprehensive review of pedestrian re-identification based on deep learning

“…Hong et al [183] supplemented the knowledge of clothes-independent shapes and improved re-ID performance. Lu et al [184] obtained the movement information of pedestrians by fusing gait and appearance features, generated robust and discriminating features for better identity re-ID. Wu et al [185] used a clothing-independent spatial attention module to eliminate the interference of clothing appearance, by obtaining information features from the body resolution module, effectively reducing the computational cost in the re-ID task.…”