A Review of Deep Learning-based Human Activity Recognition on Benchmark Video Datasets

Sharma, Vijeta; Gupta, Manjari; Pandey, Anil; Mishra, Deepti; Kumar, Ajai

doi:10.1080/08839514.2022.2093705

Cited by 48 publications

(18 citation statements)

References 91 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The video clips of the HMDB51 do not require any low-level filtering or improvement [34]. That means the dataset is ready to train the LSTM network.…”

Section: ) Data Processingmentioning

confidence: 99%

A Novel Front Door Security (FDS) Algorithm Using GoogleNet-BiLSTM Hybridization

Paula¹,

Faruqui

Mahmud

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Security has always been a significant concern since the dawn of human civilization. That is why we build houses to keep ourselves and our belongings safe. And we do not hesitate to spend a lot on front-door locks and install CCTV cameras to monitor security threats. This paper presents an innovative automatic Front Door Security (FDS) algorithm that uses Human Activity Recognition (HAR) to detect four different security threats at the front door from a real-time video feed with 73.18% accuracy. The activities are recognized using an innovative combination of GoogleNet-BiLSTM hybrid network. This network receives the video feed from the CCTV camera and classifies the activities. The proposed algorithm uses this classification to alert any attempts to break the door by kicking, punching, or hitting. Furthermore, the proposed FDS algorithm is effective in detecting gun violence at the front door, which further strengthens security. This Human Activity Recognition (HAR)-based novel FDS algorithm demonstrates the potential of ensuring better safety with 71.49% precision, 68.2% recall, and an F1-score of 0.65.

show abstract

“…The video clips of the HMDB51 do not require any low-level filtering or improvement [34]. That means the dataset is ready to train the LSTM network.…”

Section: ) Data Processingmentioning

confidence: 99%

A Novel Front Door Security (FDS) Algorithm Using GoogleNet-BiLSTM Hybridization

Paula¹,

Faruqui

Mahmud

et al. 2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…However, there are certain limitations to this method. It requires much labor to create the salient features suitable for the situation, and most frequently used feature extractors are created using a particular dataset, making them biased against databases as they cannot extract features for all purposes [126]. Furthermore, the use of conventional approaches to support ABD in ADL are insufficient because they are only effective with limited data.…”

Section: Model Is Customizable For User's Need Does Not Cover Kitchen...mentioning

confidence: 99%

A Review of Abnormal Behavior Detection in Activities of Daily Living

et al. 2023

View full text Add to dashboard Cite

Abnormal behavior detection (ABD) systems are built to automatically identify and recognize abnormal behavior from various input data types, such as sensor-based and vision-based input. As much as the attention received for ABD systems, the number of studies on ABD in activities of daily living (ADL) is limited. Owing to the increasing rate of elderly accidents in the home compound, ABD in ADL research should be given as much attention to preventing accidents by sending out signals when abnormal behavior such as falling is detected. In this study, we compare and contrast the formation of the ABD system in ADL from input data types (sensor-based input and vision-based input) to modeling techniques (conventional and deep learning approaches). We scrutinize the public datasets available and provide solutions for one of the significant issues: the lack of datasets in ABD in ADL. This work aims to guide new research to understand the field of ABD in ADL better and serve as a reference for future study of better Ambient Assisted Living with the growing smart home trend. INDEX TERMS abnormal behavior detection, activities of daily living

show abstract

“…Human motion research began in the 1850s with E. J. Marey and E. Muybridge shooting moving people [13]. Various taxonomies for recognizing movement have been proposed [6,14]. The following sections discuss HAR's two cutting-edge learning approaches (Shallow and Deep learning).…”

Section: Related Workmentioning

confidence: 99%

Leveraging Transfer Learning for Spatio-Temporal Human Activity Recognition from Video Sequences

Butt¹,

Ullah²,

Letchmunan³

et al. 2023

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

Human Activity Recognition (HAR) is an active research area due to its applications in pervasive computing, human-computer interaction, artificial intelligence, health care, and social sciences. Moreover, dynamic environments and anthropometric differences between individuals make it harder to recognize actions. This study focused on human activity in video sequences acquired with an RGB camera because of its vast range of real-world applications. It uses two-stream ConvNet to extract spatial and temporal information and proposes a fine-tuned deep neural network. Moreover, the transfer learning paradigm is adopted to extract varied and fixed frames while reusing object identification information. Six state-of-the-art pre-trained models are exploited to find the best model for spatial feature extraction. For temporal sequence, this study uses dense optical flow following the two-stream Con-vNet and Bidirectional Long Short Term Memory (BiLSTM) to capture longterm dependencies. Two state-of-the-art datasets, UCF101 and HMDB51, are used for evaluation purposes. In addition, seven state-of-the-art optimizers are used to fine-tune the proposed network parameters. Furthermore, this study utilizes an ensemble mechanism to aggregate spatial-temporal features using a four-stream Convolutional Neural Network (CNN), where two streams use RGB data. In contrast, the other uses optical flow images. Finally, the proposed ensemble approach using max hard voting outperforms state-ofthe-art methods with 96.30% and 90.07% accuracies on the UCF101 and HMDB51 datasets.

show abstract

A Review of Deep Learning-based Human Activity Recognition on Benchmark Video Datasets

Cited by 48 publications

References 91 publications

A Novel Front Door Security (FDS) Algorithm Using GoogleNet-BiLSTM Hybridization

A Novel Front Door Security (FDS) Algorithm Using GoogleNet-BiLSTM Hybridization

A Review of Abnormal Behavior Detection in Activities of Daily Living

Leveraging Transfer Learning for Spatio-Temporal Human Activity Recognition from Video Sequences

Contact Info

Product

Resources

About