Modeling Temporal Structure of Decomposable Motion Segments for Activity Classification

Abstract

Abstract. Much recent research in human activity recognition has focused on the problem of recognizing simple repetitive (walking, running, waving) and punctual actions (sitting up, opening a door, hugging). However, many interesting human activities are characterized by a complex temporal composition of simple actions. Automatic recognition of such complex actions can benefit from a good understanding of the temporal structures. We present in this paper a framework for modeling motion by exploiting the temporal structure of the human activities. In our framework, we represent activities as temporal compositions of motion segments. We train a discriminative model that encodes a temporal decomposition of video sequences, and appearance models for each motion segment. In recognition, a query video is matched to the model according to the learned appearances and motion segment decomposition. Classification is made based on the quality of matching between the motion segment classifiers and the temporal segments in the query sequence. To validate our approach, we introduce a new dataset of complex Olympic Sports activities. We show that our algorithm performs better than other state of the art methods.

Motion Based Foreground Detection and Poselet Motion Features for Action Recognition

“…Moreover, some of the actions have to be learned from a very small number of training samples. To show that the approach works on other datasets, we also evaluated it on the Olympic Sports dataset [28]. The performance is 77.5% (without poselets) and 85.5% (with poselet motion features).…”

“…The performance is 77.5% (without poselets) and 85.5% (with poselet motion features). However, the Olympic Sports dataset [28] uses a very small test set (some actions are evaluated on only 3-4 video clips). This means that the overall score for Olympic Sports can be strongly affected based on just a few test examples.…”

Source constrained clustering

“…Marszalek et al [13] report classification results of average precision 0.326 using a BoW framework and STIP features, clearly showing the difficulty of the problem. The same approach applied to action classification from YouTube videos of sport events shows that BoW approaches on real world data sets need further improvement [16]. Similarly, prior work on clustering features extracted from video sequences from the CMU Multimodal Database [4] shows that several algorithms cluster samples from the same subjects, rather than discriminate samples across action classes.…”

Unstructured human activity detection from RGBD images

“…One common approach is to use space-time features to model points of interest in video [15,6]. Several authors have supplemented these techniques by adding more information to these features [11,40,41,19,25,30]. However, this approach is only capable of classifying, rather than detecting, activities.…”