Robust human activity recognition from depth video using spatiotemporal multi-fused features

Kishore

Mathematical Problems in Engineering

2017

Extracting and recognizing complex human movements from unconstraint online video sequence is an interesting task. In this paper the complicated problem from the class is approached using unconstraint video sequences belonging to Indian classical dance forms. A new segmentation model is developed using discrete wavelet transform and local binary pattern (LBP) features for segmentation. A 2D point cloud is created from the local human shape changes in subsequent video frames. The classifier is fed with 5 types of features calculated from Zernike moments, Hu moments, shape signature, LBP features, and Haar features. We also explore multiple feature fusion models with early fusion during segmentation stage and late fusion after segmentation for improving the classification process. The extracted features input the Adaboost multiclass classifier with labels from the corresponding song (tala). We test the classifier on online dance videos and on an Indian classical dance dataset prepared in our lab. The algorithms were tested for accuracy and correctness in identifying the dance postures.

Section: Literature Surveymentioning

confidence: 99%

Indian Classical Dance Classification with Adaboost Multiclass Classifier on Multifeature Fusion

Kishore

Mathematical Problems in Engineering

2017

“…Hence, these approaches need to be robust in estimating human actions. Data driven methods with multiple feature fusion [7] with artificial intelligence models [8] are currently being explored with the increase in computing power.…”

Section: Literature Reviewmentioning

confidence: 99%

Indian Classical Dance Action Identification and Classification with Convolutional Neural Networks

Kishore

et al. 2018

Advances in Multimedia

Extracting and recognizing complex human movements from unconstrained online/offline video sequence is a challenging task in computer vision. This paper proposes the classification of Indian classical dance actions using a powerful artificial intelligence tool: convolutional neural networks (CNN). In this work, human action recognition on Indian classical dance videos is performed on recordings from both offline (controlled recording) and online (live performances, YouTube) data. The offline data is created with ten different subjects performing 200 familiar dance mudras/poses from different Indian classical dance forms under various background environments. The online dance data is collected from YouTube for ten different subjects. Each dance pose is occupied for 60 frames or images in a video in both the cases. CNN training is performed with 8 different sample sizes, each consisting of multiple sets of subjects. The remaining 2 samples are used for testing the trained CNN. Different CNN architectures were designed and tested with our data to obtain a better accuracy in recognition. We achieved a 93.33% recognition rate compared to other classifier models reported on the same dataset.

2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)

“…A Long Short Term Memory (LSTM) based approach [10] was presented to perform classification. In [11], Jalal et al present a system for activity recognition and temporal segmentation based on skeletal and silhouette features. Start and end of the activity time intervals are found comparing fitness between a non-activity model and the models of each activity built using HMMs.…”

Section: Related Workmentioning

confidence: 99%

Automatic detection of human interactions from RGB-D data for social activity classification

Coppola

Coşar

Faria

et al. 2017

Abstract-We present a system for temporal detection of social interactions. Many of the works until now have succeeded in recognising activities from clipped videos in datasets, but for robotic applications, it is important to be able to move to more realistic data. For this reason, the proposed approach temporally detects intervals where individual or social activity is occurring. Recognition of human activities is a key feature for analysing the human behaviour. In particular, recognition of social activities is useful to trigger human-robot interactions or to detect situations of potential danger. Based on that, this research has three goals: (1) define a new set of descriptors, which are able to characterise human interactions; (2) develop a computational model to segment temporal intervals with social interaction or individual behaviour; (3) provide a public dataset with RGB-D data with continuous stream of individual activities and social interactions. Results show that the proposed approach attained relevant performance with temporal segmentation of social activities.