Skeleton Motion Recognition Based on Multi-Scale Deep Spatio-Temporal Features

Hu, Kai; Ding, Yiwu; Jin, Junlan; Weng, Liguo; Xia, Min

doi:10.3390/app12031028

Cited by 30 publications

(24 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results of these two comparisons are shown in Table 3 and Table 4 , in these tables, bolded data is best. The methods used for comparison include the handcrafted feature-based method [ 35 ], RNN-based methods [ 36 , 37 , 38 ], CNN-based methods [ 39 , 40 ], and GCN-based methods [ 5 , 14 , 16 , 41 , 42 , 43 , 44 , 45 , 46 , 47 ]. The accuracy of MA-CT in CV index on NTU-RGB + D is 95.9%, and the accuracy in the CS index is 89.7%.…”

Section: Experimental Results and Analysismentioning

confidence: 99%

“…We take 2S-AGCN as the basic model that was introduced in detail in our other paper [ 5 ]. This article will briefly introduce the prominent contents.…”

Section: Proposed Methodsmentioning

confidence: 99%

“…With the rapid development of artificial intelligence algorithms, motion-recognition technology, which is an important part of artificial intelligence, is being studied for its application in many fields, such as human–computer interaction, video surveillance, film and television production, and other areas [ 1 , 2 , 3 ]. Many researchers [ 4 , 5 , 6 ] have invested a great deal of energy in this field and designed many excellent algorithms. Among them, most of the traditional algorithms use manual feature extraction, and these algorithms have made a breakthrough [ 7 ].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multiple Attention Mechanism Graph Convolution HAR Model Based on Coordination Theory

Ding

Jin

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

Human action recognition (HAR) is the foundation of human behavior comprehension. It is of great significance and can be used in many real-world applications. From the point of view of human kinematics, the coordination of limbs is an important intrinsic factor of motion and contains a great deal of information. In addition, for different movements, the HAR algorithm provides important, multifaceted attention to each joint. Based on the above analysis, this paper proposes a HAR algorithm, which adopts two attention modules that work together to extract the coordination characteristics in the process of motion, and strengthens the attention of the model to the more important joints in the process of moving. Experimental data shows these two modules can improve the recognition accuracy of the model on the public HAR dataset (NTU-RGB + D, Kinetics-Skeleton).

show abstract

Section: Experimental Results and Analysismentioning

confidence: 99%

“…We take 2S-AGCN as the basic model that was introduced in detail in our other paper [ 5 ]. This article will briefly introduce the prominent contents.…”

Section: Proposed Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multiple Attention Mechanism Graph Convolution HAR Model Based on Coordination Theory

Ding

Jin

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…As an important branch of machine learning, deep learning has achieved remarkable results in image recognition [139], semantic understanding [140], image matching [141], 3D reconstruction [142], and other tasks. The application of deep learning in computer vision can greatly ease the problems encountered by traditional methods [143]. Traditional VSLAM systems have achieved commendable results in many aspects, but there are still many challenging problems to be solved [144].…”

Section: Semantic Vslammentioning

confidence: 99%

An Overview on Visual SLAM: From Tradition to Semantic

et al. 2022

Self Cite

View full text Add to dashboard Cite

Visual SLAM (VSLAM) has been developing rapidly due to its advantages of low-cost sensors, the easy fusion of other sensors, and richer environmental information. Traditional visionbased SLAM research has made many achievements, but it may fail to achieve wished results in challenging environments. Deep learning has promoted the development of computer vision, and the combination of deep learning and SLAM has attracted more and more attention. Semantic information, as high-level environmental information, can enable robots to better understand the surrounding environment. This paper introduces the development of VSLAM technology from two aspects: traditional VSLAM and semantic VSLAM combined with deep learning. For traditional VSLAM, we summarize the advantages and disadvantages of indirect and direct methods in detail and give some classical VSLAM open-source algorithms. In addition, we focus on the development of semantic VSLAM based on deep learning. Starting with typical neural networks CNN and RNN, we summarize the improvement of neural networks for the VSLAM system in detail. Later, we focus on the help of target detection and semantic segmentation for VSLAM semantic information introduction. We believe that the development of the future intelligent era cannot be without the help of semantic technology. Introducing deep learning into the VSLAM system to provide semantic information can help robots better perceive the surrounding environment and provide people with higher-level help.

show abstract

“…HMR model can be implemented by focusing on specific human parts' motion states. Then, motions can be positioned by the constraint of the tree structure and specific motion information (Hu et al, 2022 ). Meanwhile, the spatiotemporal and graphical models can be combined to build an integrated HMR framework (Low et al, 2022 ).…”

Section: Literature Surveymentioning

confidence: 99%

Applying Deep Learning-Based Human Motion Recognition System in Sports Competition

Zhang

2022

Front. Neurorobot.

View full text Add to dashboard Cite

The exploration here intends to compensate for the traditional human motion recognition (HMR) systems' poor performance on large-scale datasets and micromotions. To this end, improvement is designed for the HMR in sports competition based on the deep learning (DL) algorithm. First, the background and research status of HMR are introduced. Then, a new HMR algorithm is proposed based on kernel extreme learning machine (KELM) multidimensional feature fusion (MFF). Afterward, a simulation experiment is designed to evaluate the performance of the proposed KELM-MFF-based HMR algorithm. The results showed that the recognition rate of the proposed KELM-MFF-based HMR is higher than other algorithms. The recognition rate at 10 video frame sampling points is ranked from high to low: the proposed KELM-MFF-based HMR, support vector machine (SVM)-MFF-based HMR, convolutional neural network (CNN) + optical flow (CNN-T)-based HMR, improved dense trajectory (IDT)-based HMR, converse3D (C3D)-based HMR, and CNN-based HMR. Meanwhile, the feature recognition rate of the proposed KELM-MFF-based HMR for the color dimension is higher than the time dimension, by up to 24%. Besides, the proposed KELM-MFF-based HMR algorithm's recognition rate is 92.4% under early feature fusion and 92.1% under late feature fusion, higher than 91.8 and 90.5% of the SVM-MFF-based HMR. Finally, the proposed KELM-MFF-based HMR algorithm takes 30 and 15 s for training and testing. Therefore, the algorithm designed here can be used to deal with large-scale datasets and capture and recognize micromotions. The research content provides a reference for applying extreme learning machine algorithms in sports competitions.

show abstract

Skeleton Motion Recognition Based on Multi-Scale Deep Spatio-Temporal Features

Cited by 30 publications

References 36 publications

Multiple Attention Mechanism Graph Convolution HAR Model Based on Coordination Theory

Multiple Attention Mechanism Graph Convolution HAR Model Based on Coordination Theory

An Overview on Visual SLAM: From Tradition to Semantic

Applying Deep Learning-Based Human Motion Recognition System in Sports Competition

Contact Info

Product

Resources

About