A Comprehensive Review of Recent Deep Learning Techniques for Human Activity Recognition

Abstract

Human action recognition is an important field in computer vision that has attracted remarkable attention from researchers. This survey aims to provide a comprehensive overview of recent human action recognition approaches based on deep learning using RGB video data. Our work divides recent deep learning-based methods into five different categories to provide a comprehensive overview for researchers who are interested in this field of computer vision. Moreover, a pure-transformer architecture (convolution-free) has outperformed its convolutional counterparts in many fields of computer vision recently. Our work also provides recent convolution-free-based methods which replaced convolution networks with the transformer networks that achieved state-of-the-art results on many human action recognition datasets. Firstly, we discuss proposed methods based on a 2D convolutional neural network. Then, methods based on a recurrent neural network which is used to capture motion information are discussed. 3D convolutional neural network-based methods are used in many recent approaches to capture both spatial and temporal information in videos. However, with long action videos, multistream approaches with different streams to encode different features are reviewed. We also compare the performance of recently proposed methods on four popular benchmark datasets. We review 26 benchmark datasets for human action recognition. Some potential research directions are discussed to conclude this survey.

Dynamic Edge Convolutional Neural Network for Skeleton-Based Human Action Recognition

“…Because skeleton data formats provide more compact information with only 3D coordinate values, they are easily manageable and can be used to build HAR methods for lightweight devices. The upcoming subsections describe state-of-the-art HAR methods introduced using 3D skeleton data modalities, as listed in Table 1 in terms of category, year of publication, method, and dataset [ 21 ].…”

Machine learning in TCM with natural products and molecules: current status and future perspectives

“…The LSTM is an advanced variant of RNN with the capability of preserving long-term dependencies by using internal feedback [ 22 ]. Essentially, the LSTM layers prevent older information from gradually vanishing [ 23 ].…”

“…Moreover, the CNN model pooling layer loses a lot of valuable information, and the lack of a memory function as well as the limited data size and high computational requirements are other shortcomings of CNN [ 41 ] (Table 1 ). The Elman RNN has shown good ability in capturing the dynamics of sequences via recurrent connection, e.g., as in natural language processing [ 22 ]. The Elman RNN is effective and shows good generalization ability and has been widely used for solving practical problems [ 21 ].…”

Triple-stream Deep Metric Learning of Great Ape Behavioural Actions

“…Since then, several networks that utilise additional modalities, such as motion saliency (Zong et al, 2021) and audio (Wang et al, 2021), have been introduced. Recently, the introduction of pose, which is critical for the perception of actions (Le et al, 2022), has shown promising results in multi-stream architectures (Hong et al, 2019;Hayakawa and Dariush, 2020;Duan et al, 2021;Li et al, 2022). In particular, the DensePose format provides an opportunity to exploit fine-grained, segmentation map-based pose representations for action recognition.…”