Optimal Deep Convolutional Neural Network with Pose Estimation for Human Activity Recognition

Abstract

Human Action Recognition (HAR) and pose estimation from videos have gained significant attention among research communities due to its application in several areas namely intelligent surveillance, human robot interaction, robot vision, etc. Though considerable improvements have been made in recent days, design of an effective and accurate action recognition model is yet a difficult process owing to the existence of different obstacles such as variations in camera angle, occlusion, background, movement speed, and so on. From the literature, it is observed that hard to deal with the temporal dimension in the action recognition process. Convolutional neural network (CNN) models could be used widely to solve this. With this motivation, this study designs a novel key point extraction with deep convolutional neural networks based pose estimation (KPE-DCNN) model for activity recognition. The KPE-DCNN technique initially converts the input video into a sequence of frames followed by a three stage process namely key point extraction, hyperparameter tuning, and pose estimation. In the keypoint extraction process an OpenPose model is designed to compute the accurate keypoints in the human pose. Then, an optimal DCNN model is developed to classify the human activities label based on the extracted key points. For improving the training process of the DCNN technique, RMSProp optimizer is used to optimally adjust the hyperparameters such as learning rate, batch size, and epoch count. The experimental results tested using benchmark dataset like UCF sports dataset showed that KPE-DCNN technique is able to achieve good results compared with benchmark algorithms like CNN, DBN, SVM, STAL, T-CNN and so on.

Human Activity Recognition Using Cascaded Dual Attention CNN and Bi-Directional GRU Framework

“…They then conducted a thorough analysis of recent approaches proposed to meet these specific requirements and categorized them accordingly. Nandagopal et al [44] proposed a novel method for activity recognition called KPE-DCNN. This technique involves several stages: the input video is first transformed into a series of frames, followed by key point extraction using a customized OpenPose model.…”

Multi-Sensor Data Fusion and CNN-LSTM Model for Human Activity Recognition System

“…Chadia Khraief et al [ 11 ] constructed a model with four independent CNNs corresponding to video data and utilized data combined with a 4D-CNN network to verify its effect on several datasets. Nandagopal et al [ 12 ] designed a novel key point extraction with a deep convolutional neural network-based pose estimation (KPE-DCNN) model to extract the key points of the human body in the image sequences converted from video data for HAR. The KPE-DCNN model outperforms other networks, such as CNN, DBN, and T-CNN, achieving an accuracy of 85.44% on the UCF dataset.…”

“…In order to better reflect the features of human activities through Range-FFT results, we visualize the Range-FFT results as time-frequency spectrograms and Range spectrograms. (1) Time-frequency spectrogram In this work, we convert the 1D millimeter wave radar signal into a two-dimensional (2D) time-frequency spectrogram with STFT (Short-time Fourier Transform) and provide it in Equation (12).…”