Histogram of oriented rectangles: A new pose descriptor for human action recognition

Abstract

Most of the approaches to human action recognition tend to form complex models which require lots of parameter estimation and computation time. In this study, we show that, human actions can be simply represented by pose without dealing with the complex representation of dynamics. Based on this idea, we propose a novel pose descriptor which we name as Histogram-of-Oriented-Rectangles (HOR) for representing and recognizing human actions in videos. We represent each human pose in an action sequence by oriented rectangular patches extracted over the human silhouette. We then form spatial oriented histograms to represent the distribution of these rectangular patches. We make use of several matching strategies to carry the information from the spatial domain described by the HOR descriptor to temporal domain. These are (i) nearest neighbor classification, which recognizes the actions by matching the descriptors of each frame, (ii) global histogramming, which extends the idea of Motion Energy Image proposed by Bobick and Davis to rectangular patches, (iii) a classifier-based approach using Support Vector Machines, and (iv) adaptation of Dynamic Time Warping on the temporal representation of the HOR descriptor. For the cases when pose descriptor is not sufficiently strong alone, such as to differentiate actions "jogging" and "running", we also incorporate a simple velocity descriptor as a prior to the pose based classification step. We test our system with different configurations and experiment on two commonly used action datasets: the Weizmann dataset and the KTH dataset. Results show that our method is superior to other methods on Weizmann dataset with a perfect accuracy rate of 100%, and is comparable to the other methods on KTH dataset with a very high success rate close to 90%. These results prove that with a simple and compact representation, we can achieve robust recognition of human actions, compared to complex representations. © 2009 Elsevier B.V. All rights reserved

Recognizing Human Actions Using Key Poses

“…Our method requires large K values to achieve good classification performance because distinct actors may perform an action in different ways. The recognition results of different methods in the literature vary between 73% [12] and 100% [7] on the Weizmann dataset and between 71.72% [15] and 93.80% [10] on the KTH dataset. Our results show that the pose information by itself is quite effective in grasping the nature of an action and sufficient to distinguish one from the others.…”

“…Ikizler et al [7] propose a "bag of rectangles" method that represents the human body as a collection of oriented rectangle patches and uses spatial oriented histograms. Thurau et al [16] extend Histogram of Oriented Gradients (HOG) based descriptor to represent pose primitives.…”

Initial-state invariant Binet-Cauchy kernels for the comparison of Linear Dynamical Systems

“…Recognition (%) Xie et al [38] 95.60 Thurau et al [39] 94.40 Ikizler et al [40] 100.00 Gorelick et al [37] 99.60 Niebles et al [41] 90.00 Ali et al [42] 95 the original Binet-Cauchy kernel. This shows the importance of having an initial-state invariant metrics as the best results are achieved using the Martin distance and the Binet-Cauchy determinant kernel which are both invariant w.r.t.initial states of the dynamical systems.…”

A Review on Machine Learning Algorithms on Human Action Recognition

“…Ikizler and Duygulu [21] proposed another posture descriptor called histogram of oriented rectangles (HOR) for action recognition. They represented every human posture in an action succession with oriented rectangular patches separated over the human outline, which then framed spatial oriented histograms to represent the circulation of these rectangular patches.…”