Robust 3D Action Recognition with Random Occupancy Patterns

Abstract

We study the problem of action recognition from depth sequences captured by depth cameras, where noise and occlusion are common problems because they are captured with a single commodity camera. In order to deal with these issues, we extract semi-local features called random occupancy pattern (ROP) features, which employ a novel sampling scheme that effectively explores an extremely large sampling space. We also utilize a sparse coding approach to robustly encode these features. The proposed approach does not require careful parameter tuning. Its training is very fast due to the use of the high-dimensional integral image, and it is robust to the occlusions. Our technique is evaluated on two datasets captured by commodity depth cameras: an action dataset and a hand gesture dataset. Our classification results are superior to those obtained by the state of the art approaches on both datasets.

Evolutionary joint selection to improve human action recognition with RGB-D devices

“…So, our optimisation approach obtains very good results using a recognition method that was not specifically designed for RGB-D devices. Therefore, the next step will be to apply it to the best and more recent methods in the state of the art (Wang et al, 2012a;Wang et al, 2012b) in order to study the level of optimisation achieved.…”

“…The concatenation of the three HOG serves as input feature to a linear SVM classifier. Wang et al (2012a) treat an action sequence as a 4D shape and propose random occupancy pattern features, which are extracted from randomly sampled 4D sub-volumes with different sizes and at different locations. These features are robust to noise and less sensitive to occlusions.…”

Action recognition based on binocular vision

“…Ali and Aggarwal 12 define an action boundary by using a feature vector that contains three angles to the main part of the human body. Hanjalic 13 15 used semi-Markov model to achieve the action segmentation. For the RGBD method, the depth of the human body can be obtained by face detection or head and shoulder detection, and then the human body position can be extracted using the depth information.…”

Natural Action Recognition Using Invariant 3D Motion Encoding

“…The use of 3D data has removed the need for this step in much recent work recognising actions in constrained environments, due to the simplicity of segmenting the actor (for example by using the Kinect's user mask) [19,4,8]. This enables complex "volumetric" descriptions of the actors body over time [34,32,31,23]. However, for "in the wild" action recognition this is not the case as it generally remains impossible to segment the actor reliably, due to noisy 3D data, cluttered environments, and scenes containing multiple people.…”