Action recognition based on a mixture of RGB and depth based skeleton

Abstract

In this paper, we study how different skeleton extraction methods affect the performance of action recognition. As shown in previous work skeleton information can be exploited for action recognition. Nevertheless, skeleton detection problem is already hard and very often it is difficult to obtain reliable skeleton information from videos. In this paper, we compare two skeleton detection methods: the depthmap based method used with Kinect camera and RGB based method that uses Deep Convolutional Neural Networks. In order to balance the pros and cons of mentioned skeleton detection methods w.r.t. action recognition task, we propose a fusion of classifiers trained based on each skeleton detection method. Such fusion lead to performance improvement. We validate our approach on CAD-60 and MSRDailyActivity3D, achieving state-of-the-art results.

Spatio-Temporal Grids for Daily Living Action Recognition

“…So, in these datasets spatio-temporal grids have contributed the most in achieving a competitive global recognition rate as compared to the state-of-the-art (results are in table 2 and 3). The performance of [8] similar to our performance on CAD-60, is not consistent on other datasets. Method [22] though outperforms our framework on CAD-120, is time expensive and the brute force enumeration over all settings of the latent variables cause extra computational cost.…”

“…But most of these techniques fail to discriminate the daily living activities because of its challenges. Authors in [6,8] extract spatial features from different parts of the body by first cropping out the parts from RGB frames using skeleton joint information. But, cropping different parts of the body and resizing them into 224×224 so as to feed the network is not a robust technique.…”

A New Hybrid Architecture for Human Activity Recognition from RGB-D Videos

“…This can be done by a technique of regression on the confusion matrix in the training phase. [23] 70.26 VA-LSTM [22] 79.4 CMN [24] 80.8 STA-hands [2] 82.5 Glimpse Clouds [3] 86.6 Proposed Method 87.09 Proposed Method (with I3D) 92.20 Table 4: Recognition Accuracy comparison for CAD-60 , CAD-120, MSRDai-lyActivity3D (Performance of baseline is taken from [8,12,7] respectively) and NTU-RGB+D dataset.…”

“…Early models extract CNN features from video frames and aggregates them with pooling for classifying by SVM. The authors in [5,8] use different body part patches to extract features from a convolutional network in order to recognize actions. The requirement to introduce spatio-temporal relationship in videos motivated the authors in [4] to use 3D convolutions.…”

A real-time human-robot interaction framework with robust background invariant hand gesture detection

“…In our work we utilize OpenPose [53,52], to extract skeletal joint coordinates, as in [56,57]. This library returns 2D skeletal coordinates (x i , y i , c i ), for i =…”