Not All Parts Are Created Equal: 3D Pose Estimation by Modeling Bi-Directional Dependencies of Body Parts

Abstract

Not all the human body parts have the same degree of freedom (DOF) due to the physiological structure. For example, the limbs may move more flexibly and freely than the torso does. Most of the existing 3D pose estimation methods, despite the very promising results achieved, treat the body joints equally and consequently often lead to larger reconstruction errors on the limbs. In this paper, we propose a progressive approach that explicitly accounts for the distinct DOFs among the body parts. We model parts with higher DOFs like the elbows, as dependent components of the corresponding parts with lower DOFs like the torso, of which the 3D locations can be more reliably estimated. Meanwhile, the high-DOF parts may in turn impose a constraint on where the low-DOF ones lie. As a result, parts with different DOFs supervise one another, yielding physically constrained and plausible pose-estimation results. To further facilitate the prediction of the high-DOF parts, we introduce a pose-attribute estimation, where the relative location of a limb joint with respect to the torso, which has the least DOF of a human body, is explicitly estimated and further fed to the joint-estimation module. The proposed approach achieves very promising results, outperforming the state of the art on several benchmarks.

Pointless Pose: Part Affinity Field-Based 3D Pose Estimation without Detecting Keypoints

“…The MPLORE results are shown in Table 2. We compare our method with three state-of-the-art methods [18,32,34]. Our method achieves the best results on 14 of the 15 activities, and the best averaged result.…”

“…Pavlakos et al [33] predicted the depth of human joints using manually annotated ordinal depth supervision by a ranking loss. Wang et al [34] defined the pose attributes as intermediate image cues to reduce the ambiguity in lifting 2D pose into 3D space. These methods requires that all the human keypoints are present in the image, which is unfortunately too strong for real-world application scenarios, where out-of-image absences of body joints frequently occur and thus collapse the 3D estimation results.…”

“…The best score is marked in bold. We achieve the best results across all the activities except for Walking, which is only slightly worse than[34] 11…”

“…Following many previous works [18,21,24,[32][33][34]38,39], we use a 2D pose dataset MPII [11] and a 3D pose dataset Human3.6M [12] to train the network. In training phase, each mini-batch is randomly sampled from the two datasets with equal probability.…”

“…A case study on images with various geometric occlusions. We compare the results with three state-of-the-art methods: (a) [18], (b) [32], and (c) [34]. Our method is robust under missing key points, rectangular and circle-occlusions, as well as partial-blurring.…”