Evaluation of video activity localizations integrating quality and quantity measurements

Abstract

Evaluating the performance of computer vision algorithms is classically done by reporting classification error or accuracy, if the problem at hand is the classification of an object in an image, the recognition of an activity in a video or the categorization and labeling of the image or video. If in addition the detection of an item in an image or a video, and/or its localization are required, frequently used metrics are Recall and Precision, as well as ROC curves. These metrics give quantitative performance values which are easy to understand and to interpret even by non-experts. However, an inherent problem is the dependency of quantitative performance measures on the quality constraints that we need impose on the detection algorithm. In particular, an important quality parameter of these measures is the spatial or spatio-temporal overlap between a ground-truth item and a detected item, and this needs to be taken into account when interpreting the results.We propose a new performance metric addressing and unifying the qualitative and quantitative aspects of the performance measures. The performance of a detection and recognition algorithm is illustrated intuitively by performance graphs which present quantitative performance values, like Recall, Precision and F-Score, depending on quality constraints of the detection. In order to compare the performance of different computer vision algorithms, a representative single performance measure is computed from the graphs, by integrating out all quality parameters. The evaluation method can be applied to different types of activity detection and recognition algorithms. The performance metric has been tested on several activity recognition algorithms participating in the ICPR 2012 HARL competition.

Automatic Human Activity Segmentation and Labeling in RGBD Videos

“…An initial research was conducted to analyze several datasets from different sources like LIRIS (Laboratoire d'InfoRmatique en Image et Systèmes d'information) dataset [19], CMU (Carnegie Mellon University) MoCap dataset 5 , MSR-Action3D and MSRDailyActivity3D dataset [13] and verify it's suitability to our problem. All these datasets contain only isolated actions, and for our task we require sequences of actions.…”

Weakly supervised activity analysis with spatio-temporal localisation

“…The results also suggest that it outperforms the original MIML model [33], the state-ofthe-art weakly supervised approaches [25,24,18], as well as fully supervised methods [4,29,22,32] in the literature across the three datasets. The paper is organised as follows: Section 2 provides a review of related work for MIML techniques and weakly supervised action detection; Section 3 details the feature representation of video under the weakly supervised setting; Section 4 formulates the proposed framework and introduces the generation of instances and bags in the setting of weakly supervised action detection; Section 5 describes the experiments, such as data and implementation details; Section 6 demonstrates results and analysis of the experiments; 3 finally Section 7 concludes this work and points out possible future work.…”

Simulated Lidar Images of Human Pose using a 3DS Max Virtual Laboratory

“…With the release of low-cost range cameras such as Kinect (Microsoft), new datasets in the form of 3D depth images were generated for the purpose of human action analysis and recognition. Some examples of public available ones are MSR action 3D [1] and MSR daily activity [2], LIRIS human activity [3], and UT Kinect action [4], etc. These depth images were acquired in much closer ranges (< 4m) than the typical operational range of low-grade commercial LIDARs (80~100m).…”