Graph convolutional networks and LSTM for first-person multimodal hand action recognition

Li, Rui; Wang, Hongyu

doi:10.1007/s00138-022-01328-4

Cited by 9 publications

(9 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These compact data are less affected by complex backgrounds and viewpoint changes. Graph Convolutional Networks (GCNs) are the best way to use skeleton data [27]. Xu et al [28] proposed a two-stream model based on the human skeleton and scene images.…”

Section: Skeleton-based Action Recognition: Viewpointsmentioning

confidence: 99%

“…NTU Dataset: During the performance assessment, we track the ordinary CV and CS conventions proposed by [32]. Our approach is compared with Deep Learning methodologies that incorporate RNNs or CNNs and skeleton data [17,27,28,35,39]. The use of handcrafted elements in some traditional approaches [54,55].…”

Section: Compared With Other Approachesmentioning

confidence: 99%

“…This dataset has hundreds of viewpoints, making action recognition difficult. VA-RNN(aug.) and VA-CNN(aug.) outpace standard arrangements and more RNN-and CNN-based systems that apply innovative techniques [17,27,28,35,39] for instance, attention [34,39].…”

Section: Compared With Other Approachesmentioning

confidence: 99%

See 2 more Smart Citations

RETRACTED ARTICLE: Automatically human action recognition (HAR) with view variation from skeleton means of adaptive transformer network

et al. 2023

View full text Add to dashboard Cite

Human Action Recognition (HAR) using skeletons has become increasingly appealing to a growing number of researchers in recent years. It is particularly challenging to recognize actions when they are captured from different angles because there are so many variations in their representations. This paper proposes an automatic strategy for determining virtual observation viewpoints that are based on learning and data-driven to solve the problem of view variation throughout an act. Our VA-CNN and VA-RNN networks, which use convolutional and recurrent neural networks with the Long Short-term Memory Article Title (LSTM), offer an alternative to the conventional method of reorienting skeletons according to a human-defined earlier benchmark. Using the unique view adaption module, each network first identifies the best observation perspectives and then transforms the skeletons for endto-end detection with the main classification network based on those viewpoints. Using the skeletons of different perspectives, the suggested view adaptive models can provide significantly more consistent virtual viewpoints. By removing views, the models allow networks to learn action-specific properties more efficiently. Furthermore, we developed a two-stream scheme (referred to as VA-fusion) that integrates the performance of two networks to obtain an improved prediction. Random rotation of skeletal sequences is used to avoid overfitting during training and improve the reliability of view adaption models. An extensive experiment demonstrates that our proposed view-adaptive networks outperform existing solutions on five challenging benchmarks.

show abstract

Section: Skeleton-based Action Recognition: Viewpointsmentioning

confidence: 99%

Section: Compared With Other Approachesmentioning

confidence: 99%

See 1 more Smart Citation

RETRACTED ARTICLE: Automatically human action recognition (HAR) with view variation from skeleton means of adaptive transformer network

et al. 2023

View full text Add to dashboard Cite

show abstract

Section: Compared With Other Approachesmentioning

confidence: 99%

Section: Compared With Other Approachesmentioning

confidence: 99%

Automatically Human Action Recognition (HAR) with View Variation from Skeleton Means of Adaptive Transformer Network

Mehmood

Chen

Akbar

et al. 2022

Preprint

View full text Add to dashboard Cite

Human Action Recognition (HAR) using skeletons has become increasingly appealing to a growing number of researchers in recent years. It is particularly challenging to recognize actions when they are captured from different angles because there are so many variations in their representations. This paper proposes an automatic strategy for determining virtual observation viewpoints that are based on learning and data-driven to solve the problem of view variation throughout an act. Our VA-CNN and VA-RNN networks, which use convolutional and recurrent neural networks with the Long Short-term Memory (LSTM), offer an alternative to the conventional method of reorienting skeletons according to a human-defined earlier benchmark. Using the unique view adaption module, each network first identifies the best observation perspectives and then transforms the skeletons for end-to-end detection with the main classification network based on those viewpoints. Using the skeletons of different perspectives, the suggested view adaptive models can provide significantly more consistent virtual viewpoints. By removing views, the models allow networks to learn action-specific properties more efficiently. Furthermore, we developed a two-stream scheme (referred to as VA-fusion) that integrates the performance of two networks to obtain an improved prediction. Random rotation of skeletal sequences is used to avoid overfitting during training and improve the reliability of view adaption models. An extensive experiment demonstrates that our proposed view-adaptive networks outperform existing solutions on five challenging benchmarks.

show abstract

A multilayer human motion prediction perceptron by aggregating repetitive motion

Geng,

Yang,

Jiao

et al. 2023

Machine Vision and Applications

View full text Add to dashboard Cite

Graph convolutional networks and LSTM for first-person multimodal hand action recognition

Cited by 9 publications

References 61 publications

RETRACTED ARTICLE: Automatically human action recognition (HAR) with view variation from skeleton means of adaptive transformer network

RETRACTED ARTICLE: Automatically human action recognition (HAR) with view variation from skeleton means of adaptive transformer network

Automatically Human Action Recognition (HAR) with View Variation from Skeleton Means of Adaptive Transformer Network

A multilayer human motion prediction perceptron by aggregating repetitive motion

Contact Info

Product

Resources

About