Global Co-Occurrence Feature and Local Spatial Feature Learning for Skeleton-Based Action Recognition

Abstract

Recent progress on skeleton-based action recognition has been substantial, benefiting mostly from the explosive development of Graph Convolutional Networks (GCN). However, prevailing GCN-based methods may not effectively capture the global co-occurrence features among joints and the local spatial structure features composed of adjacent bones. They also ignore the effect of channels unrelated to action recognition on model performance. Accordingly, to address these issues, we propose a Global Co-occurrence feature and Local Spatial feature learning model (GCLS) consisting of two branches. The first branch, based on the Vertex Attention Mechanism branch (VAM-branch), captures the global co-occurrence feature of actions effectively; the second, based on the Cross-kernel Feature Fusion branch (CFF-branch), extracts local spatial structure features composed of adjacent bones and restrains the channels unrelated to action recognition. Extensive experiments on two large-scale datasets, NTU-RGB+D and Kinetics, demonstrate that GCLS achieves the best performance when compared to the mainstream approaches.

Skeleton Graph-Neural-Network-Based Human Action Recognition: A Survey

“…Instead of simple self-attention, such as what AGCN used, J. Xie et al [ 127 ] integrated channel attention module (CAM) into their Vertex Attention Mechanism (VAM) to extract the global co-occurrence features of actions. The CAM generates channel weights by performing a fast 1DCNN in adaptive kernel size.…”

“…Methods, such as [ 49 , 73 , 74 , 100 , 120 , 126 , 127 , 131 ], assumed that features in different channels have various importance, and thus they attempted to balance the importance of each channel while inferring, known as channel-wise attention.…”

“…Green texts annotate the methods measured by ‘Params(M)’, and blue texts annotate ‘FLOPs (G)/Action’. The numbers around dots denote [ 13 , 14 , 50 , 54 , 55 , 57 , 59 , 62 , 73 , 77 , 80 , 82 , 85 , 86 , 89 , 110 , 116 , 117 , 121 , 127 , 130 , 135 , 141 , 183 ] respectively in ascending order. Number 94 [ 54 ] is a remarkable one, with both relatively low complexity and small model size.…”

“…Table A2 summarizes the action list and methods performed on each dataset. The most popular compared methods across datasets are [ 13 , 14 , 50 , 54 , 55 , 57 , 59 , 62 , 73 , 77 , 80 , 82 , 85 , 86 , 89 , 110 , 116 , 117 , 121 , 127 , 130 , 135 , 141 , 183 ], among which, Ref. [ 54 ] is a remarkable one on NTU RGB+D cross-subject, with both low complexity and small model size but good performance.…”

A Dance Somersault Pose Recognition Model Using Multifeature Fusion Algorithm

“…Accurate description of motion posture by effective feature information is the key factor to improve the recognition effect of human motion posture. Dance somersault skill is an important index to evaluate dancers' basic skills and comprehensive abilities, and identifying dance somersault posture is of great significance to art teaching and research [14]. Therefore, this paper constructs a dance somersault pose recognition model based on multifeature fusion.…”