EgoFish3D: Egocentric 3D Pose Estimation From a Fisheye Camera via Self-Supervised Learning

Liu, Yuxuan; Yang, Jun; Gu, Xiao; Chen, Yijun; Guo, Yao; Yang, Guang‐Zhong

doi:10.1109/tmm.2023.3242551

Cited by 11 publications

(1 citation statement)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specifically, the body parts further from the camera, such as the lower limb, are shown much smaller than the real size, while the shoulder and upper limb views are enlarged. Existing methods aim to tackle the challenge with reprojection methods to alleviate the distortion effect [Liu et al 2023;Xu et al 2019;Zhang et al 2021] or extensively train the network with large synthetic datasets [Akada et al 2022;Tome et al 2019]. However, these approaches overlook the potential value of the perspective factor, which can be useful information conveying the intensity of the 3D effect.…”

Section: Introductionmentioning

confidence: 99%

Ego3DPose: Capturing 3D Cues from Binocular Egocentric Views

Kang,

Lee,

Zhang

et al. 2023

SIGGRAPH Asia 2023 Conference Papers

View full text Add to dashboard Cite

We present Ego3DPose, a highly accurate binocular egocentric 3D pose reconstruction system. The binocular egocentric setup offers practicality and usefulness in various applications, however, it remains largely under-explored. It has been suffering from low pose estimation accuracy due to viewing distortion, severe selfocclusion, and limited field-of-view of the joints in egocentric 2D images. Here, we notice that two important 3D cues, stereo correspondences, and perspective, contained in the egocentric binocular input are neglected. Current methods heavily rely on 2D image features, implicitly learning 3D information, which introduces biases towards commonly observed motions and leads to low overall accuracy. We observe that they not only fail in challenging occlusion cases but also in estimating visible joint positions. To address these challenges, we propose two novel approaches. First, we design a two-path network architecture with a path that estimates pose per limb independently with its binocular heatmaps. Without full-body information provided, it alleviates bias toward trained full-body distribution. Second, we leverage the egocentric view of body limbs, which exhibits strong perspective variance (e.g., a

show abstract