Multi-Term Attention Networks for Skeleton-Based Action Recognition

Abstract

The same action takes different time in different cases. This difference will affect the accuracy of action recognition to a certain extent. We propose an end-to-end deep neural network called “Multi-Term Attention Networks” (MTANs), which solves the above problem by extracting temporal features with different time scales. The network consists of a Multi-Term Attention Recurrent Neural Network (MTA-RNN) and a Spatio-Temporal Convolutional Neural Network (ST-CNN). In MTA-RNN, a method for fusing multi-term temporal features are proposed to extract the temporal dependence of different time scales, and the weighted fusion temporal feature is recalibrated by the attention mechanism. Ablation research proves that this network has powerful spatio-temporal dynamic modeling capabilities for actions with different time scales. We perform extensive experiments on four challenging benchmark datasets, including the NTU RGB+D dataset, UT-Kinect dataset, Northwestern-UCLA dataset, and UWA3DII dataset. Our method achieves better results than the state-of-the-art benchmarks, which demonstrates the effectiveness of MTANs.

Kinect Azure–Based Accurate Measurement of Dynamic Valgus Position of the Knee—A Corrigible Predisposing Factor of Osteoarthritis

“…Kinect validation, a subtask of this study, was performed in order to be sure that the Kinect system is appropriate and reliable device for medical examination rooms, orthopedic physiotherapy practices or even gyms. Most studies in the literature with Kinect-based movement tracking systems use the earlier V2 camera [34][35][36], whereas in the present study we used the completely redesigned Kinect Azure, which has an advanced Time-of-Flight depth sensor and AI algorithm to identify joint midpoints. The validation of this camera system-even if it is limited to the single-leg squat-is a valuable resource for other investigators who are experimenting with the novel capabilities of Kinect Azure in various research settings.…”

Multi-Modal Multi Feature Assisted Human Action Recognition through Deep Learning

“…They employed totally five classifiers namely (1) "LogDet divergence based metric learning with triplet constraints (LDMLT)", (2) "Bidirectional long short-term memory network (BiLSTM)", (3) "Fully convolutional network (FCN)", (4) "DTW with city block distance (DTW-CBD)", and (5) "DTW with Euclidean distance (DTW-ED)". X. Diao et al [21] suggested a new RNN model named as "Multi-term attention networks (MTANs)" for HAR from body postures. MTANs can extract the temporal features are different scales which consists of MTA-RNN and ST-CNN.…”