Self-Supervised Gait Encoding with Locality-Aware Attention for Person Re-Identification

Abstract

<jats:p>Gait-based person re-identification (Re-ID) is valuable for safety-critical applications, and using only 3D skeleton data to extract discriminative gait features for person Re-ID is an emerging open topic. Existing methods either adopt hand-crafted features or learn gait features by traditional supervised learning paradigms. Unlike previous methods, we for the first time propose a generic gait encoding approach that can utilize unlabeled skeleton data to learn gait representations in a self-supervised manner. Specifically, we first propose to introduce self-supervision by learning to reconstruct input skeleton sequences in reverse order, which facilitates learning richer high-level semantics and better gait representations. Second, inspired by the fact that motion's continuity endows temporally adjacent skeletons with higher correlations (“locality”), we propose a locality-aware attention mechanism that encourages learning larger attention weights for temporally adjacent skeletons when reconstructing current skeleton, so as to learn locality when encoding gait. Finally, we propose Attention-based Gait Encodings (AGEs), which are built using context vectors learned by locality-aware attention, as final gait representations. AGEs are directly utilized to realize effective person Re-ID. Our approach typically improves existing skeleton-based methods by 10-20% Rank-1 accuracy, and it achieves comparable or even superior performance to multi-modal methods with extra RGB or depth information.</jats:p>

Person Re-Identification from different views based on dynamic linear combination of distances

“…We can conclude that the used data are on small datasets (11 persons) which do not promote machine learning to obtain good results. From table 1, we can deduce that the Proposed Method (PM) is more accurate in rank 1 to many recent skeleton-based methods ( [30] and [33]) and it even outperforms the deep learning methods ( [55] and [34]) on many datasets.…”

“…This dataset is composed of a "Still" sequence and a "Walking" sequence collected in different days and in different locations while the subjects have been dressed differently. It is worth noting that we have used the full training set and the "Walking" testing set that contains dynamic skeleton data, similarly to the work of [34].…”

SM-SGE: A Self-Supervised Multi-Scale Skeleton Graph Encoding Framework for Person Re-Identification

“…In contrast, skeleton-based models exploit 3D coordinates of key body joints to characterize human body and motion, which are usually robust to factors such as view and body shape changes [9]. Despite that skeleton data have been extensively studied in action and motion related tasks [15], it is still an open challenge to extract discriminative body and motion features with 3D skeletons for person Re-ID [24]. In this sense, this work aims to construct a systematic framework from three aspects to tackle the skeleton-based person Re-ID task.…”

“…In human walking, body components usually possess different internal relations, which could carry unique and recognizable patterns [21,30]. Recent works like [16,24,25] typically encode body-joint trajectory or pre-defined pose descriptors into a feature vector for skeleton representation learning, while they rarely explore the inherent relations between different body joints or components. For example, adjacent body joints "knee" and "foot" are strongly correlated in walking, while they enjoy different degrees of collaboration with their corresponding limb-level component "leg".…”

“…(3) Multi-scale skeleton dynamics modeling. Existing 3D skeleton based Re-ID methods usually model skeleton dynamics from the trajectory of body joints [24,25] or the sequence of predefined joint features (𝑒.𝑔., pairwise joint distances and pose descriptors [16]). Since these methods learn skeleton motion at a fixed scale of body joints, they lack the flexibility to capture motion patterns at various levels.…”