Human Action Recognition by Learning Spatio-Temporal Features With Deep Neural Networks

Wang, Lei; Xu, Yangyang; Cheng, Jun; Xia, Haiying; Yin, Jianqin; Wu, Jiaji

doi:10.1109/access.2018.2817253

Cited by 95 publications

(40 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Accuracy (Souly and Shah, 2016) [21] 85.10% (Wang et al, 2009) [16] 85.60% (Le et al, 2011) [1] 86.50% (Kovashka and Grauman, 2010) [17] 87.20% (Wang et al, 2011) [18] 89.10% (Weinzaepfel et al, 2015) [22] 90.50% (Abdulmunem et al, 2016) [20] 90.90% (Ravanbakhsh et al, 2015) [37] 88.10% (Wang et al, 2018) [39] 91.89% (Zhou et al, 2017) [28] 90.00% Basic method 85.30% Proposed method 92.00% UCF-11 Methods Accuracy (Hasan et al, 2014) [2] 54.50% (Liu et al, 2009) [43] 71.20% (Ikizler-Cinbis et al, 2010) [36] 75.20% (Wang et al, 2011) [18] 84.20% (Sharma et al, 2015) [27] 84.90% (Cho et al, 2014) [19] 88.00% (Ravanbakhsh et al, 2015) [37] 77.10% (Wang et al, 2018) [39] 98.76% (Gammulle et al, 2017) [38] 89.20% (Gilbert et al, 2017) [3] 86.70% Basic method 82.40% Proposed method 92.40%…”

Section: Ucf Sport Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Human activity recognition by using MHIs of frame sequences

ZEBHI¹

2020

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

A motion history image (MHI) is a temporal template that collapses temporal motion information into a single image in which intensity is a function of recency of motion. In recent years, the popularity of deep learning architectures for human activity recognition has encouraged us to explore the effectiveness of combining them and MHIs. Based on this, two new methods are introduced in this paper. In the first method, which is called the basic method, each video splits into N groups of consecutive frames, and the MHI is calculated for each group. Transfer learning with the fine-tuning technique is used for classifying these temporal templates. The experimental results show that some misclassification errors are created because of the similarities between these temporal templates; these errors can be corrected by detecting specific objects in the scenes. Thus, spatial information consisting of a single frame is also added to the second method, called the proposed method. By converting video classification problems into image classification problems in the proposed method, less memory is needed and the time complexity is greatly reduced. They are implemented and compared with state-of-the-art approaches on two data sets. The results show that the proposed method significantly outperforms the others. It achieves recognition accuracies of 92% and 92.4% for the UCF Sport and UCF-11 action data sets, respectively.

show abstract

Section: Ucf Sport Methodsmentioning

confidence: 99%

“…Wang et al [39] proposed a lightweight architecture for video action recognition, which consisted of a CNN, LSTM, and attention model. They used the convolution model to extract two kinds of features (semantic, spatial) for each frame, followed by FC-LSTM with their temporal-wise attention model.…”

Section: Introductionmentioning

confidence: 99%

Human activity recognition by using MHIs of frame sequences

ZEBHI¹

2020

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

show abstract

“…The ConvLSTM network [124] was designed based on the LSTM by adding convolution operation, it can consider the correlation of time and space in the video at the same time, and effectively integrate time and space features, and it has been applied in action recognition [125,126] and gesture recognition [127,128]. Courtney and Sreenivas [129] replaced the convolution layer in ResNet with the ConvLSTM layer to expand the learning time and space characteristics of ResNet structure and used CTC to train the network.…”

Section: Other Deep Learning Networkmentioning

confidence: 99%

A Survey of Research on Lipreading Technology

et al. 2020

View full text Add to dashboard Cite

“…The success of the deep learning methods in the image processing tasks [15] and action recognition [15], [16], [18] task, motivated the researchers to apply these methods in the case of the abnormal activity recognition. Consequently, a deep auto-encoder based approach [5]- [7] has been proposed to learn the features of the normal activities automatically, but generalization of these methods for real-world scenarios is difficult.…”

Section: Related Workmentioning

confidence: 99%

3D ResNet with Ranking Loss Function for Abnormal Activity Detection in Videos

Dubey

Boragule

Jeon

2019

2019 International Conference on Control, Automation and Information Sciences (ICCAIS)

View full text Add to dashboard Cite

Abnormal activity detection is one of the most challenging tasks in the field of computer vision. This study is motivated by the recent state-of-art work of abnormal activity detection, which utilizes both abnormal and normal videos in learning abnormalities with the help of multiple instance learning by providing the data with video-level information. In the absence of temporal-annotations, such a model is prone to give a false alarm while detecting the abnormalities. For this reason, in this paper, we focus on the task of minimizing the false alarm rate while performing an abnormal activity detection task. The mitigation of these false alarms and recent advancement of 3D deep neural network in video action recognition task collectively give us motivation to exploit the 3D ResNet in our proposed method, which helps to extract spatial-temporal features from the videos. Afterwards, using these features and deep multiple instance learning along with the proposed ranking loss, our model learns to predict the abnormality score at the video segment level. Therefore, our proposed method 3D deep Multiple Instance Learning with ResNet (MILR) along with the new proposed ranking loss function achieves the best performance on the UCF-Crime benchmark dataset, as compared to other state-of-art methods. The effectiveness of our proposed method is demonstrated on the UCF-Crime dataset.

show abstract

Human Action Recognition by Learning Spatio-Temporal Features With Deep Neural Networks

Cited by 95 publications

References 32 publications

Human activity recognition by using MHIs of frame sequences

Human activity recognition by using MHIs of frame sequences

A Survey of Research on Lipreading Technology

3D ResNet with Ranking Loss Function for Abnormal Activity Detection in Videos

Contact Info

Product

Resources

About