DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation

Abstract

This paper considers the task of articulated human pose estimation of multiple people in real world images. We propose an approach that jointly solves the tasks of detection and pose estimation: it infers the number of persons in a scene, identifies occluded body parts, and disambiguates body parts between people in close proximity of each other. This joint formulation is in contrast to previous strategies, that address the problem by first detecting people and subsequently estimating their body pose. We propose a partitioning and labeling formulation of a set of body-part hypotheses generated with CNN-based part detectors. Our formulation, an instance of an integer linear program, implicitly performs non-maximum suppression on the set of part candidates and groups them to form configurations of body parts respecting geometric and appearance constraints. Experiments on four different datasets demonstrate state-of-the-art results for both single person and multi person pose estimation 1 .

Stacked Hourglass Networks for Human Pose Estimation

“…[28][29][30] Also, there is the more challenging task of simultaneous annotation of multiple people [17,31]. In addition, there is work like that of Oliveira et al [32] that performs human part segmentation based on fully convolutional networks [23].…”

“…Their graphical model learns typical spatial relationships between joints. Others have recently tackled this in similar ways [17,20,25] with variations on how to approach unary score generation and pairwise comparison of adjacent joints. Chen et al [25] cluster detections into typical orientations so that when their classifier makes predictions additional information is available indicating the likely location of a neighboring joint.…”

“…Recent pose estimation systems [15][16][17][18][19][20] have universally adopted ConvNets as their main building block, largely replacing hand-crafted features and graphical models; this strategy has yielded drastic improvements on standard benchmarks [1,21,22].…”

Action recognition based on a mixture of RGB and depth based skeleton

“…Most approaches [22,6,28], for multi-person pose estimation have used a top down strategy where the person is detected first and then on the detected regions, poses are estimated. There are some approaches which uses bottom up approach as in [21] that jointly labels part detection candidates and associated them to individual people. This approach does not rely on person detection but involves solving an integer linear programming over fully connected graph which is an NP-hard problem.…”

Multi-person Pose Estimation with Local Joint-to-Person Associations

“…This is mainly due to the availability of deep learning based methods for detecting joints [1][2][3][4][5]. While earlier approaches in this direction [4,6,7] combine the body part detectors with tree structured graphical models, more recent methods [1][2][3][8][9][10] demonstrate that spatial relations between joints can be directly learned by a neural network without the need of an additional graphical model. These approaches, however, assume that only a single person is visible in the image and the location of the person is known a-priori.…”