Spatio-Temporal Features based Human Action Recognition using Convolutional Long Short-Term Deep Neural Network

Abstract

Recognition of human intention is crucial and challenging due to subtle motion patterns of a series of action evolutions. Understanding of human actions is the foundation of many applications, i.e., human robot interaction, smart video monitoring and autonomous driving etc. Existing deep learning methods use either spatial or temporal features during training. This research focuses on developing a lightweight method using both spatial and temporal features to predict human intention correctly. This research proposes Convolutional Long Short-Term Deep Network (CLSTDN) consists of Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). CNN uses Inception-ResNet-v2 to classify object specific class categories by extracting spatial features and RNN uses Long Short-Term Memory (LSTM) for final prediction based on temporal features. Proposed method was validated on four challenging benchmark dataset, i.e., UCF Sports, UCF-11, KTH and UCF-50. Performance of the proposed method was evaluated using seven performance metrics, i.e., accuracy, precision, recall, f-measure, error rate, loss and confusion matrix. Proposed method showed better results comparing with existing research results. Proposed method is expected to encourage researchers to use in future for real time implications to predict human intentions more robustly.

Research on Image Semantic Segmentation Based on Hybrid Cascade Feature Fusion and Detailed Attention Mechanism

“…The application of Deep CNNs to image semantic segmentation technology has great significance for improving the application of this network in scene understanding and classification [10], [11], [12], [13], [14], [15]. Although the current research progress has made a lot of achievements, the approaches on image semantic segmentation still have the problem that some details are lost, which leads to poor segmentation effect of small-scale object and unclear segmentation of object boundary under complex background conditions [16], [17], [18], [19].…”