Learning motion-difference features using Gaussian restricted Boltzmann machines for efficient human action recognition

Abstract

This is the accepted version of the paper.This version of the publication may differ from the final published version. Abstract-Learning visual words from video frames is challenging because deciding which word to assign to each subset of frames is a difficult task. For example, two similar frames may have different meanings in describing human actions such as starting to run and starting to walk. In order to associate richer information to vector-quantization and generate visual words, several approaches have been proposed recently that use complex algorithms to extract or learn spatio-temporal features from 3-D volumes of video frames. In this paper, we propose an efficient method to use Gaussian RBM for learning motiondifference features from actions in videos. The difference between two video frames is defined by a subtraction function of one frame by another that preserves positive and negative changes, thus creating a simple spatio-temporal saliency map for an action. This subtraction function removes, by construction, the common shapes and background images that should not be relevant for action learning and recognition, and highlights the movement patterns in space, making it easier to learn the actions from such saliency maps using shallow feature learning models such as RBMs. In the experiments reported in this paper, we used a Gaussian restricted Boltzmann machine to learn the actions from saliency maps of different motion images. Despite its simplicity, the motion-difference method achieved very good performance in benchmark datasets, specifically the Weizmann dataset (98.81%) and the KTH dataset (88.89%). A comparative analysis with hand-crafted and learned features using similar classifiers indicates that motion-difference can be competitive and very efficient. Permanent repository link

Human action recognition based on discriminative supervoxels

“…KTH YOUTUBE Liu [9] 93.8% 71.2% Color STIPs [6] -78.6% Dense trajectories [11] 94.2% 83.5% Motion word [12] -88.9% DSAP [17] -76.5% Motion Difference [16] 88.89% -Interest Point Detector [5] 93.5% -Our method 95.4% 89.5% …”

“…In works [13] [14] [15], the human body regions are extracted as ROI by background subtraction and motion tracking. Tran et al [16] learns the motion difference between frames with Gaussian Restricted Boltzmann Machines. A dynamic structure preserving map is proposed in Cai et al [17].…”