Middle-Level Representation for Human Activities Recognition: The Role of Spatio-Temporal Relationships

Yuan, Fei; Prinet, V.; Yuan, Junsong

doi:10.1007/978-3-642-35749-7_13

Cited by 16 publications

(18 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several pedestrians are present in the videos as well, preventing the recognition. The UT-Interaction dataset was used for the human activity recognition contest (SDHA 2010) [14], and it has been tested by several state-of-the-art methods [17,18,19].…”

Section: Datasetmentioning

confidence: 99%

Human activity prediction: Early recognition of ongoing activities from streaming videos

Ryoo

2011

2011 International Conference on Computer Vision

486

443

View full text Add to dashboard Cite

In this paper, we present a novel approach of human activity prediction. Human activity prediction is a probabilistic process of inferring ongoing activities from videos only containing onsets (i.e. the beginning part) of the activities. The goal is to enable early recognition of unfinished activities as opposed to the after-the-fact classification of completed activities. Activity prediction methodologies are particularly necessary for surveillance systems which are required to prevent crimes and dangerous activities from occurring. We probabilistically formulate the activity prediction problem, and introduce new methodologies designed for the prediction. We represent an activity as an integral histogram of spatio-temporal features, efficiently modeling how feature distributions change over time. The new recognition methodology named dynamic bag-of-words is developed, which considers sequential nature of human activities while maintaining advantages of the bag-of-words to handle noisy observations. Our experiments confirm that our approach reliably recognizes ongoing activities from streaming videos with a high accuracy.

show abstract

Section: Datasetmentioning

confidence: 99%

Human activity prediction: Early recognition of ongoing activities from streaming videos

Ryoo

2011

2011 International Conference on Computer Vision

486

443

View full text Add to dashboard Cite

show abstract

“…half observation full observation Integral BoW [6] 65% 81.7% Dynamic BoW [6] 70% 85 % Cuboid + Bayesian [6] 25% 71.7% Cuboid + SVMs [7] 31.7% 85% BP-SVM [8] -83.3% Pose 'Doublet' [12] -79.17% Mid-level [4] -78.2% Our proposed 80% 91.7% Table 1: Comparison of classification results on UTInteraction.…”

Section: Methodsmentioning

confidence: 97%

Predicting human activities using spatio-temporal structure of interest points

Yuan

Liu

2012

Proceedings of the 20th ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

Early recognition and prediction of human activities are of great importance in video surveillance, e.g., by recognizing a criminal activity at its beginning stage, it is possible to avoid unfortunate outcomes. We address early activity recognition by developing a Spatial-Temporal Implicit Shape Model (STISM), which characterizes the space-time structure of the sparse local features extracted from a video. The early recognition of human activities is accomplished by pattern matching through STISM. To enable efficient and robust matching, we propose a new random forest structure, called multi-class balanced random forest, which makes a good trade-off between the balance of the trees and the discriminative abilities. The prediction is done simultaneously for multiple classes, which saves both the memory and computational cost. The experiments show that our algorithm significantly outperforms the state of the arts for the human activity prediction problem.

show abstract

“…Accuracy D-BoW [27] 85.0% I-BoW [27] 81.7% Cuboid SVM [26] 85.0% BP-SVM [19] 83.3% Cuboid/Bayesian [27] 71.7% DP-SVM [34] 14.6% Yu et al [50] 83.3% Yuan et al [51] 78.2% Waltisberg et al [41] 88.0% Ours 90.0% Diff. to State-of-the-Art +2.0% Table 8.…”

Section: Methodsmentioning

confidence: 99%

Encouraging LSTMs to Anticipate Actions Very Early

Aliakbarian

Saleh

Salzmann

et al. 2017

2017 IEEE International Conference on Computer Vision (ICCV)

158

205

View full text Add to dashboard Cite

In contrast to the widely studied problem of recognizing an action given a complete sequence, action anticipation aims to identify the action from only partially available videos. As such, it is therefore key to the success of computer vision applications requiring to react as early as possible, such as autonomous navigation. In this paper, we propose a new action anticipation method that achieves high prediction accuracy even in the presence of a very small percentage of a video sequence. To this end, we develop a multi-stage LSTM architecture that leverages context-aware and action-aware features, and introduce a novel loss function that encourages the model to predict the correct class as early as possible. Our experiments on standard benchmark datasets evidence the benefits of our approach; We outperform the state-of-the-art action anticipation methods for early prediction by a relative increase in accuracy of 22.0% on JHMDB-21, 14.0% on UT-Interaction and 49.9% on UCF-101.

show abstract

Middle-Level Representation for Human Activities Recognition: The Role of Spatio-Temporal Relationships

Cited by 16 publications

References 27 publications

Human activity prediction: Early recognition of ongoing activities from streaming videos

Human activity prediction: Early recognition of ongoing activities from streaming videos

Predicting human activities using spatio-temporal structure of interest points

Encouraging LSTMs to Anticipate Actions Very Early

Contact Info

Product

Resources

About