WildGait: Learning Gait Representations from Raw Surveillance Streams

Abstract

The use of gait for person identification has important advantages such as being non-invasive, unobtrusive, not requiring cooperation and being less likely to be obscured compared to other biometrics. Existing methods for gait recognition require cooperative gait scenarios, in which a single person is walking multiple times in a straight line in front of a camera. We address the challenges of real-world scenarios in which camera feeds capture multiple people, who in most cases pass in front of the camera only once. We address privacy concerns by using only motion information of walking individuals, with no identifiable appearance-based information. As such, we propose a self-supervised learning framework, WildGait, which consists of pre-training a Spatio-Temporal Graph Convolutional Network on a large number of automatically annotated skeleton sequences obtained from raw, real-world surveillance streams to learn useful gait signatures. We collected and compiled the largest pretraining dataset to date of anonymized walking skeletons called Uncooperative Wild Gait, containing over 38k tracklets of anonymized walking 2D skeletons. We make the dataset available to the research community. Our results surpass the current state-of-the-art pose-based gait recognition solutions. Our proposed method is reliable in training gait recognition methods in unconstrained environments, especially in settings with scarce amounts of annotated data.

Gait Recognition with Self-Supervised Learning of Gait Features Based on Vision Transformers

“…As a result, there are not many methods that use SSL for gait recognition. Among the first to investigate the application of SSL learning to gait recognition was WildGait [ 35 ]. In their manuscript, the authors created the novel Uncooperative Wild Gait (UWG) dataset, in which gait representations are automatically annotated by recognizing skeletal sequences of individuals.…”

“…They use the horizontal pyramid mapping (HPM) [ 3 ] and micro-motion template builder (MTB) [ 4 ] spatio-temporal backbones, which are specifically designed for the gait recognition task. As in WildGait [ 35 ], the proposed approaches use CNN as the backbone network.…”

Learning Gait Representations with Noisy Multi-Task Learning

“…This noninvasive method of biometric processing has found significant applications across several domains, including security and surveillance [9] and healthcare [12]. Gait patterns offer insights into the identity of a person [9], demographics [4], emotions [24] and mental state [10].…”

“…Skeleton-based methods, where the human body is represented as a set of interconnected joints [13], offer several advantages as they are less sensitive to changes in clothing and lighting conditions, and they provide a more robust and interpretable feature set [25]. Current state-of-the-art methods for skeleton-based gait analysis [26], [11] process sequences of skeletons as spatial-temporal graphs, using models such as ST-GCN [29] and MS-G3D [21]. Despite their effectiveness, these methods have a significant drawback: they are black-boxes that provide no inherent insight into their decision process.…”

“…Moreover, they showed that the model can provide insights into its decision process through activation maps. Cosma et al [11] used an ST-GCN to learn discriminative gait features in a self-supervised manner. Catruna et al [5] used a hierarchical transformer that utilizes the anatomical structure of the human body to construct discriminative gait features.…”