HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human Motion

Abstract

While research on articulated human motion and pose estimation has progressed rapidly in the last few years, there has been no systematic quantitative evaluation of competing methods to establish the current state of the art. We present data obtained using a hardware system that is able to capture synchronized video and ground-truth 3D motion. The resulting HUMANEVA datasets contain multiple subjects performing a set of predefined actions with a number of repetitions. On the order of 40,000 frames of synchronized motion capture and multi-view video (resulting in over one quarter million image frames in total) were collected at 60 Hz with an additional 37,000 time instants of pure motion capture data. A standard set of error measures is defined for evaluating both 2D and 3D pose estimation and tracking algorithms. We also describe a baseline algorithm for 3D articulated tracking that uses a relatively standard Bayesian framework with optimization in the form of Sequential Importance Resampling and Annealed Particle Filtering. In the context of this baseline algorithm we explore a variety of likelihood functions, prior models of human motion and the effects of algorithm parameters. Our experiments suggest that image observation models and motion priors play important roles in performance, and that in a multiview laboratory environment, where initialization is available, Bayesian filtering tends to perform well. The datasets and the software are made available to the research community. This infrastructure will support the development of new articulated motion and pose estimation algorithms, will provide a baseline for the evaluation and comparison of new methods, and will help establish the current state of the art in human pose estimation and tracking.

An Optimization Based Framework for Human Pose Estimation in Monocular Videos

“…13 salient points on human body: head center, right shoulder, right elbow, right hand, left shoulder, left elbow, left hand, right hip, right knee, right foot (ankle), left hip, left knee, left foot (ankle) were manually marked for all videos in the corpus. We build upon the pose error metric proposed in [21] and define the following pose evaluation metrics for each vignette in the corpus: (a) Average error per frame as in (5), (b) Average error per marker per frame (D aepmpf ) (average of (5) for number of markers) , (c) Average error for different markers per frame as in (6).…”

Staying Well Grounded in Markerless Motion Capture

“…This is already pretty accurate but the black curves show an even better and smoother tracking with a deviation of up to just 2mm. In the second experiment we took a sequence of the HumanEVA-II benchmark [12]. Here a surface model, calibrated image sequences, and background images are provided.…”

“…It depicts the unconstrained results in red and the constrained results in black. Table 1 compares the errors (automatically evaluated [12]). Overall the tracking has been improved remarkably using the additional ground plane constraint.…”

“…exp(θ jξ j )X i (12) and we can generate a set of equations forcing the transformed point X i to stay close to X i :…”

“…Moreover, it is verified to which extent such constraints improve the tracking performance. In particular, we show results on the recent HumanEVA-II [12] benchmark, which involves a quantitative error analysis.…”