Make Skeleton-based Action Recognition Model Smaller, Faster and Better

Yang, Fan; Wu, Yang; Sakti, Sakriani; Nakamura, Satoshi

doi:10.1145/3338533.3366569

Cited by 141 publications

(70 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Some existing studies have been considering the model complexity problem. The study [32] constructs a lightweight network with CNN-based blocks, which is not as accurate as GCN models. The work [35] adopts a complex data preprocessing strategy, whose inputs include positions, velocities, frame indexes and joint types.…”

Section: Related Workmentioning

confidence: 99%

Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition

Song

Zhang

Shan

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

272

166

View full text Add to dashboard Cite

One essential problem in skeleton-based action recognition is how to extract discriminative features over all skeleton joints. However, the complexity of the State-Of-The-Art (SOTA) models of this task tends to be exceedingly sophisticated and over-parameterized, where the low efficiency in model training and inference has obstructed the development in the field, especially for large-scale action datasets. In this work, we propose an efficient but strong baseline based on Graph Convolutional Network (GCN), where three main improvements are aggregated, i.e., early fused Multiple Input Branches (MIB), Residual GCN (ResGCN) with bottleneck structure and Part-wise Attention (PartAtt) block. Firstly, an MIB is designed to enrich informative skeleton features and remain compact representations at an early fusion stage. Then, inspired by the success of the ResNet architecture in Convolutional Neural Network (CNN), a ResGCN module is introduced in GCN to alleviate computational costs and reduce learning difficulties in model training while maintain the model accuracy. Finally, a PartAtt block is proposed to discover the most essential body parts over a whole action sequence and obtain more explainable representations for different skeleton action sequences. Extensive experiments on two large-scale datasets, i.e., NTU RGB+D 60 and 120, validate that the proposed baseline slightly outperforms other SOTA models and meanwhile requires much fewer parameters during training and inference procedures, e.g., at most 34 times less than DGNN, which is one of the best SOTA methods. CCS CONCEPTS • Computing methodologies → Activity recognition and understanding.

show abstract

Section: Related Workmentioning

confidence: 99%

Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition

Song

Zhang

Shan

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

272

166

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

“…The use of CNN for online signal recognition had been studied for the task of action recognition in [19] and [20]. In [19], the authors use three types of inputs relative to gesture as input to a CNN. Each input is treated separately (in a branch) of the network and the features extracted are concatenated before being given as input to a last CNN which performs the classification.…”

Section: Related Workmentioning

confidence: 99%

Integrating Writing Dynamics in CNN for Online Children Handwriting Recognition

Corbillé

Fromont

Anquetil

et al. 2020

2020 17th International Conference on Frontiers in Handwriting Recognition (ICFHR)

View full text Add to dashboard Cite

Online handwriting recognition is challenging but an already well-studied topic. However, recent advances in the development of convolutional neural networks (CNN) make us believe that these networks could still improve the state of the art especially in the much more challenging context of online children handwritten letters recognition. This is because, children handwriting is, at an early stage of learning, approximate and includes deformed letters. To evaluate the potential of these networks, we study the early and late fusions of different input channels that can provide a CNN with information about the handwriting dynamics in addition to the static image of the characters. The experiments on a real children handwriting dataset with 27 000 characters acquired in primary schools, show that using multiple channels with CNN, improves the accuracy performance of different CNN architectures and different fusion settings for character recognition.

show abstract

“…Estimating human pose is the key to analyzing human behavior. HPE is the basic research in computer vision, which can be applied to many applications, such as Human-computer interaction, human action recognition [1][2][3][4], intelligent security, motion capture, and action detection [5].…”

Section: Introductionmentioning

confidence: 99%

3D Human Pose Estimation Based on a Fully Connected Neural Network With Adversarial Learning Prior Knowledge

Liu

Gao

2021

Front. Phys.

View full text Add to dashboard Cite

3D human pose estimation is more and more widely used in the real world, such as sports guidance, limb rehabilitation training, augmented reality, and intelligent security. Most existing human pose estimation methods are designed based on an RGB image obtained by one optical sensor, such as a digital camera. There is some prior knowledge, such as bone proportion and angle limitation of joint hinge motion. However, the existing methods do not consider the correlation between different joints from multi-view images, and most of them adopt fixed spatial prior constraints, resulting in poor generalizations. Therefore, it is essential to build a multi-view image acquisition system using optical sensors and customized algorithms for a 3D reconstruction of the human pose in the image. Inspired by generative adversarial networks (GAN), we used a data-driven method to learn the implicit spatial prior information and classified joints according to the natural connection characteristics. To accelerate the proposed method, we proposed a fully connected network with skip connections and used the SMPL model to make the 3D human body reconstruction. Experimental results showed that compared with other state-of-the-art methods, the joints’ average error of the proposed method was the smallest, which indicated the best performance. Moreover, the running time of the proposed method was 1.3 seconds per frame, which may not meet real-time requirements, but is still much faster than most existing methods.

show abstract

Make Skeleton-based Action Recognition Model Smaller, Faster and Better

Cited by 141 publications

References 34 publications

Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition

Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition

Integrating Writing Dynamics in CNN for Online Children Handwriting Recognition

3D Human Pose Estimation Based on a Fully Connected Neural Network With Adversarial Learning Prior Knowledge

Contact Info

Product

Resources

About