2D Articulated Human Pose Estimation and Retrieval in (Almost) Unconstrained Still Images

Abstract

We present a technique for estimating the spatial layout of humans in still images-the position of the head, torso and arms. The theme we explore is that once a person is localized using an upper body detector, the search for their body parts can be considerably simplified using weak constraints on position and appearance arising from that detection. Our approach is capable of estimating upper body pose in highly challenging uncontrolled images, without prior knowledge of background, clothing, lighting, or the location and scale of the person in the image. People are only required to be upright and seen from the front or the back (not side). We evaluate the stages of our approach experimentally using ground truth layout annotation on a variety of challenging material, such as images from the PASCAL VOC 2008 challenge and video frames from TV shows and feature films. We also propose and evaluate techniques for searching a video dataset for people in a specific pose. To this end, we develop three new pose descriptors and compare their classification and retrieval performance to two baselines built on state-of-the-art object detection models.

Has My Algorithm Succeeded? An Evaluator for Human Pose Estimators

“…To avoid detecting the same person twice, each face detection is then regressed into the coordinate frame of the upper-body detector and suppressed if it overlaps substantially with any upper-body detection. As shown in [6] this combination yields a higher detection rate at the same false-positive rate, compared to using either component alone.…”

“…Here, we use the publicly available detector of [6]. It combines an upper-body detector based on the part-based model of [8] and a face detector [18].…”

“…Table 2 gives the percentage of samples where pose is estimated accurately, i.e. the CPC between the estimated and ground-truth stickmen is < 0.3 In all cases, we use the implementations provided by the authors [5,16,2,22] and all methods are given the same detection windows [6] as preprocessing. Both measures agree on the relative ranking of the methods: Yang and Ramanan [22] performs best, followed by Sapp et al [16], Eichner and Ferrari [5] and then by Andriluka et al [2].…”

“…The performance is evaluated under two regimes: (A) only where the predicted HPE corresponds to one of the annotations. Since the images are fairly completely annotated, any upper body detection window [6] which does correspond to an annotation is considered a false positive. In this regime such false positives are ignored; (B) all predictions are evaluated, including false-positives.…”

Pose Estimation and Segmentation of People in 3D Movies

“…However, these works are limited to cases involving isolated people, and extending them to situations with multiple interacting people is not straightforward. Recently, a model for joint reasoning about poses of multiple upright people has been proposed in [12]. This framework does not output segmentations of people, but can be adapted to do so.…”