Graph Based Skeleton Motion Representation and Similarity Measurement for Action Recognition

Wang, Pei; Yuan, Chunfeng; Hu, Weiming; Li, Bing; Zhang, Yanning

doi:10.1007/978-3-319-46478-7_23

Cited by 91 publications

(83 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We achieved the best recognition accuracy by SPMF Inception-ResNet-222 network configuration with a total average accuracy of 98.56%. This result outperforms many previous studies [16,23,7,10,9,24,25,26,27]. For the NTU-RGB+D dataset [11], we achieved an accuracy of 78.89% on crosssubject evaluation and 86.15% on cross-view evaluation as shown in Table 2.…”

Section: Implementation Detailsmentioning

confidence: 48%

Skeletal Movement to Color Map: A Novel Representation for 3D Action Recognition with Inception Residual Networks

Pham

Khoudour

Crouzil

et al. 2018

2018 25th IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

We propose a novel skeleton-based representation for 3D action recognition in videos using Deep Convolutional Neural Networks (D-CNNs). Two key issues have been addressed: First, how to construct a robust representation that easily captures the spatial-temporal evolutions of motions from skeleton sequences. Second, how to design D-CNNs capable of learning discriminative features from the new representation in a effective manner. To address these tasks, a skeletonbased representation, namely, SPMF (Skeleton Pose-Motion Feature) is proposed. The SPMFs are built from two of the most important properties of a human action: postures and their motions. Therefore, they are able to effectively represent complex actions. For learning and recognition tasks, we design and optimize new D-CNNs based on the idea of Inception Residual networks to predict actions from SPMFs. Our method is evaluated on two challenging datasets including MSR Action3D and NTU-RGB+D. Experimental results indicated that the proposed method surpasses state-of-the-art methods whilst requiring less computation.

show abstract

Section: Implementation Detailsmentioning

confidence: 48%

Skeletal Movement to Color Map: A Novel Representation for 3D Action Recognition with Inception Residual Networks

Pham

Khoudour

Crouzil

et al. 2018

2018 25th IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

show abstract

“…We use dropout (0.5) and data augmentation to avoid overfitting. All regularization parameters were Method Accuracy Grassmann Manifold [48] 88.5 Histogram of 3D Joints [67] 90.9 Riemannian Manifold [17] 91.5 Key-Pose-Motifs [59] 93.5 LARP + mfPCA [2] 94.8 Action snippets [58] 96.5 ST LSTM + Trust Gates [36] 97.0 Lie Group [57] 97.1 Graph-based [62] 97.4 ST-NBNN [64] 98.0 SCK + DCK [32] 98.2 DPRL + GCNN [52] 98.5 GCA-LSTM (direct) [37] 98.5 CNN + Kernel Feature Maps [56] 98.9 GCA-LSTM (stepwise) [37] 99.0 CNN + LSTM [41] 99.0 KRP FS [12] 99.0 DeepGRU 100.0 determined via cross-validation on a subset of the training data. Across all experiments we use three types of data augmentation: (1) random scaling with a factor 2 of ±0.3, (2) random translation with a factor of ±1, (3) synthetic sequence generation with gesture path stochastic resampling (GPSR) [54].…”

Section: Discussionmentioning

confidence: 99%

DeepGRU: Deep Gesture Recognition Utility

Maghoumi

LaViola

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We propose DeepGRU, a novel end-to-end deep network model informed by recent developments in deep learning for gesture and action recognition, that is streamlined and device-agnostic. DeepGRU, which uses only raw skeleton, pose or vector data is quickly understood, implemented, and trained, and yet achieves state-of-the-art results on challenging datasets. At the heart of our method lies a set of stacked gated recurrent units (GRU), two fully-connected layers and a novel global attention model. We evaluate our method on seven publicly available datasets, containing various number of samples and spanning over a broad range of interactions (full-body, multi-actor, hand gestures, etc.). In all but one case we outperform the state-of-the-art pose-based methods. For instance, we achieve a recognition accuracy of 84.9% and 92.3% on cross-subject and crossview tests of the NTU RGB+D dataset respectively, and also 100% recognition accuracy on the UT-Kinect dataset. While DeepGRU works well on large datasets with many training samples, we show that even in the absence of a large number of training data, and with as little as four samples per class, DeepGRU can beat traditional methods specifically designed for small training sets. Lastly, we demonstrate that even without powerful hardware, and using only the CPU, our method can still be trained in under 10 minutes on small-scale datasets, making it an enticing choice for rapid application prototyping and development.

show abstract

“…Lie Group [14] 97.1 2014 LARP+mfPCA [66] 94.9 2015 SPGK [13] 97.4 2016 ST-NBNN [16] 98.0 2017 Bi-LSTM [22] 96.9 2018 ST-LSTM(Tree) + Trust Gate [23] 97.0 2018 DPRL [34] 98.5 2018 GR-GCN (Bone only) 96.9 GR-GCN (Bone + Intra-connection) 97.4 Complete GR-GCN model 98.5…”

Section: Methods Accuracy Yearmentioning

confidence: 99%

“…Lie Group [14] 90.9 2014 LARP+mfPCA [66] 89.7 2015 Rolling Rotations [67] 91.4 2016 SPGK [13] 91.6 2016 Transion Forests [68] 94.2 2017 MIMTL [69] 95.3 2017 Bi-LSTM [22] 93.0 2018 Deep STGC K [33] 99.1 2018 GR-GCN (Bone only) 95.5 GR-GCN (Bone + Intra-connection) 95.6 Complete GR-GCN model 98.4 effectiveness of the proposed graph construction, in which the temporal connectivities are vital.…”

Section: Methods Accuracy Yearmentioning

confidence: 99%

Optimized Skeleton-based Action Recognition via Sparsified Graph Regression

Gao

Tang

et al. 2019

Proceedings of the 27th ACM International Conference on Multimedia

173

105

View full text Add to dashboard Cite

With the prevalence of accessible depth sensors, dynamic human body skeletons have attracted much attention as a robust modality for action recognition. Previous methods model skeletons based on RNN or CNN, which has limited expressive power for irregular skeleton joints. While graph convolutional networks (GCN) have been proposed to address irregular graphstructured data, the fundamental graph construction remains challenging. In this paper, we represent skeletons naturally on graphs, and propose a graph regression based GCN (GR-GCN) for skeleton-based action recognition, aiming to capture the spatio-temporal variation in the data. As the graph representation is crucial to graph convolution, we first propose graph regression to statistically learn the underlying graph from multiple observations. In particular, we provide spatio-temporal modeling of skeletons and pose an optimization problem on the graph structure over consecutive frames, which enforces the sparsity of the underlying graph for efficient representation. The optimized graph not only connects each joint to its neighboring joints in the same frame strongly or weakly, but also links with relevant joints in the previous and subsequent frames. We then feed the optimized graph into the GCN along with the coordinates of the skeleton sequence for feature learning, where we deploy high-order and fast Chebyshev approximation of spectral graph convolution. Further, we provide analysis of the variation characterization by the Chebyshev approximation. Experimental results validate the effectiveness of the proposed graph regression and show that the proposed GR-GCN achieves the state-of-the-art performance on the widely used NTU RGB+D, UT-Kinect and SYSU 3D datasets.

show abstract

Graph Based Skeleton Motion Representation and Similarity Measurement for Action Recognition

Cited by 91 publications

References 31 publications

Skeletal Movement to Color Map: A Novel Representation for 3D Action Recognition with Inception Residual Networks

Skeletal Movement to Color Map: A Novel Representation for 3D Action Recognition with Inception Residual Networks

DeepGRU: Deep Gesture Recognition Utility

Optimized Skeleton-based Action Recognition via Sparsified Graph Regression

Contact Info

Product

Resources

About