Multi-stream part-fused graph convolutional networks for skeleton-based gait recognition

Wang, Likai; Chen, Jinyan; Chen, Zhenghang; Liu, Yuxin; Yang, Haolin

doi:10.1080/09540091.2022.2026294

Cited by 14 publications

(10 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, we compare the proposed two-branch neural network to additional state-of-the-art gait recognition methods with the same experimental settings systematically and comprehensively, including GaitGraph [9], MS-Gait [11], GaitSet [5] and GaitPart [35]. Among that, the first two methods belong to model-based and the last two methods belong to appearance-based.…”

Section: Comparisons With State-of-art Methodsmentioning

confidence: 99%

“…sequence learning [31], image captioning [32] and action recognition [33], we introduce an attention module in this section. The preliminary version of this paper [11] presented a SE block which only works on the channel dimension. However, intuitively, in addition to the focus on the channel dimension, the distinguishment of information in spatial and temporal dimension is equally critical.…”

Section: Attention Mechanismmentioning

confidence: 99%

“…For further improvement of performance, we also fuse part-level features proposed in our previous work [11] in this paper to capture fine-grained information. In addition, we apply a multi-stream framework [11] that input the joint, bone and motion data to three GCN streams with the same model architecture separately and fuse all the output feature maps as the final gait representations, which has been proven to significantly boost the individual recognition.…”

Section: Model-based Branchmentioning

confidence: 99%

“…In the experiments, we compare the accuracy of the model without STC-Att and the model with STC-Att embedded after GCN blocks. Notably, according to [11], channel-wise attention for joint stream is just beneficial to normal walking but harmful to other states, so we merely focus on spatial-wise and temporal-wise attention for joint stream in this paper. The results are presented in Table 2, which show the impact of attention strategy.…”

Section: Effectiveness Of Stc-attmentioning

confidence: 99%

“…But in practice, it is important to directly capture joint dependency between two nodes that are far apart, such as right wrist and left ankle, the movement direction and patterns of which are closely related when people walking. To achieve this, existing methods [11] apply excessive adjacency matrixes for different joint pair connections, which causes high training costs. This means that the method suffers from the balance problem between training difficulty and the effective fusion of important information from distant joints, i.e., the distant-joint connection problem.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Combining the Silhouette and Skeleton Data for Gait Recognition

Wang¹,

Han²,

Chen³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Gait recognition, a promising long-distance biometric technology, has aroused intense interest in computer vision. Existing works on gait recognition can be divided into appearance-based methods and modelbased methods, which extract features from silhouettes and skeleton data, respectively. However, since appearance-based methods are greatly affected by clothing changing and carrying condition, and model-based methods are limited by the accuracy of pose estimation approaches, gait recognition remains challenging in practical applications. In order to integrate the merits of such two approaches, a two-branch neural network (NN)-based model is proposed in this paper. The method contains two branches, namely a CNN-based branch taking silhouettes as input and a GCN-based branch taking skeletons as input. In addition, two modifications are introduced into the GCNbased branch to boost the performance. First, we present a simple fully connected graph convolution operator to integrate multi-scale graph convolutions and relieve dependence on natural connections. Second, we deploy an attention module named STC-Att after each GCN block to learn spatial, temporal and channel-wise attention simultaneously. We evaluated the proposed two-branch neural network on the CASIA-B dataset. The experimental results show that our method achieves state-of-the-art performance in various conditions.

show abstract