Embodied Scene-aware Human Pose Estimation

Abstract

Figure 1: From a monocular video input in a known scene, we estimate 3D absolute body pose by controlling a simulated character to match 2D observation in a sequential manner.

Physically Plausible 3D Human-Scene Reconstruction From Monocular RGB Image Using an Adversarial Learning Approach

“…Scene-guided human pose and motion estimation: Recently, few methods have leveraged the scene information for accurate human reconstruction and motion predictions [17], [18], [19], [20], [21]. These methods optimize the human pose based on its contacts and collisions with the given ground-truth scene.…”

“…Hassan et al [20] propose character animation with challenging scene interactions using an adversarial discriminator that assess the realism of the human motion in the context of the scene. Luo et al [21] propose human 3D pose estimation by utilizing motion imitation in prescanned 3D scenes. Few approaches have utilized the ground-truth scene information for accurate placement of the human [22] or generating human [23] in the given scene.…”

T109 Evaluation of the GEM® Premier™ 5000 with intelligent quality management 2 (IQM®2) at Beijing Tsinghua Changgung Hospital, Tsinghua University (Beijing, China)

“…[46,56,68] adopt physics-based motion optimization to adjust the body positions and orientations. Other works adopt motion imitation in the physical simulator to estimate human pose from optical non-line-of-sight imaging system [23], sparse inertial sensors [68], and videos [71,72,31,32,75]. [74] integrate the imitation policy trained in a physics simulator into the sampling process of the diffusion model.…”

“…[74] integrate the imitation policy trained in a physics simulator into the sampling process of the diffusion model. However, most of these works [56,31,75,32,68,74] still rely on a manually designed residual force [73] at the root joint to compensate for the dynamics mismatch between the physics model and real humans, which does not apply to real robot control.…”