Improving Robustness and Accuracy via Relative Information Encoding in 3D Human Pose Estimation

Abstract

Most of the existing 3D human pose estimation approaches mainly focus on predicting 3D positional relationships between the root joint and other human joints (local motion) instead of the overall trajectory of the human body (global motion). Despite the great progress achieved by these approaches, they are not robust to global motion, and lack the ability to accurately predict local motion with a small movement range. To alleviate these two problems, we propose a relative information encoding method that yields positional and temporal enhanced representations. Firstly, we encode positional information by utilizing relative coordinates of 2D poses to enhance the consistency between the input and output distribution. The same posture with different absolute 2D positions can be mapped to a common representation. It is beneficial to resist the interference of global motion on the prediction results. Second, we encode temporal information by establishing the connection between the current pose and other poses of the same person within a period of time. More attention will be paid to the movement changes before and after the current pose, resulting in better prediction performance on local motion with a small movement range. The ablation studies validate the effectiveness of the proposed relative information encoding method. Besides, we introduce a multi-stage optimization method to the whole framework to further exploit the positional and temporal enhanced representations. Our method outperforms state-of-the-art methods on two public datasets. Code is available at https://github.com/paTRICK-swk/Pose3D-RIE.

Markerless vs. Marker-Based Gait Analysis: A Proof of Concept Study

“…To evaluate the accuracy of the full architecture, we computed the MPJPE across all the detected keypoints and obtained an error of 23.65 millimeters, again comparable to the one obtained in [15] (e.g., 19.5 millimeters on the same dataset, however with fewer keypoints-the feet were excluded) and also comparable with the error obtained in the best performing recent works about 3D pose estimation (between 19 and 30 millimeters) [40][41][42][43].…”

Ray3D: ray-based 3D human pose estimation for monocular absolute 3D localization

“…The majority of lifting based approaches work on rootrelative 3D human pose estimation [3,9,12,27,28,34,36,39,51,53,55]. [28] is the pioneer work that introduces the lifting design.…”

“…[28] is the pioneer work that introduces the lifting design. [3,34,36,53] exploit temporal information to improve the 3D pose estimation accuracy, especially for the occluded cases. Pose ambiguities can be partially resolved by exploiting the temporal context.…”

“…Pose ambiguities can be partially resolved by exploiting the temporal context. Shan et al [36] propose to encode relative positional and temporal enhanced representations, and this approach achieves excellent rootrelative 3D pose accuracy. Inspired by [36], we encode relative 3D normalized rays to improve the root-relative pose model.…”

“…With this simple design, our Ray3D approach achieves stable performance regardless of camera intrinsic parameter changes. Inspired by Videopose [34] and RIE [36], we fuse 3D rays from consecutive frames by using temporal convolution in order to further resolve the ambiguity introduced by occlusion and to improve accuracy. This temporal fusion mechanism stabilizes the output and generates more accurate 3D locations.…”