Self-Supervised 3D Human Pose Estimation with Multiple-View Geometry

Bouazizi, Arij; Wiederer, Julian; Kreßel, Ulrich; Belagiannis, Vasileios

doi:10.1109/fg52635.2021.9667074

Cited by 20 publications

(6 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhou et al [ 33 ] augmented a 2D pose estimator with a depth regression sub-network and jointly trained both sub-nets with 2D and 3D labels to fully exploit the correlation between 2D pose and depth estimation sub-tasks. Other works like [ 34 ] looked to exploit multi-view information only during training. Hua et al [ 35 ] proposed a U-shaped cross-view graph convolution network (GCN) that was trained without 3D labels.…”

Section: Related Workmentioning

confidence: 99%

PosturePose: Optimized Posture Analysis for Semi-Supervised Monocular 3D Human Pose Estimation

Amadi,

Agam

2023

Sensors

View full text Add to dashboard Cite

One motivation for studying semi-supervised techniques for human pose estimation is to compensate for the lack of variety in curated 3D human pose datasets by combining labeled 3D pose data with readily available unlabeled video data—effectively, leveraging the annotations of the former and the rich variety of the latter to train more robust pose estimators. In this paper, we propose a novel, fully differentiable posture consistency loss that is unaffected by camera orientation and improves monocular human pose estimators trained with limited labeled 3D pose data. Our semi-supervised monocular 3D pose framework combines biomechanical pose regularization with a multi-view posture (and pose) consistency objective function. We show that posture optimization was effective at decreasing pose estimation errors when applied to a 2D–3D lifting network (VPose3D) and two well-studied datasets (H36M and 3DHP). Specifically, the proposed semi-supervised framework with multi-view posture and pose loss lowered the mean per-joint position error (MPJPE) of leading semi-supervised methods by up to 15% (−7.6 mm) when camera parameters of unlabeled poses were provided. Without camera parameters, our semi-supervised framework with posture loss improved semi-supervised state-of-the-art methods by 17% (−15.6 mm decrease in MPJPE). Overall, our pose models compete favorably with other high-performing pose models trained under similar conditions with limited labeled data.

show abstract

Section: Related Workmentioning

confidence: 99%

PosturePose: Optimized Posture Analysis for Semi-Supervised Monocular 3D Human Pose Estimation

Amadi,

Agam

2023

Sensors

View full text Add to dashboard Cite

show abstract

“…For gesture recognition of police officers, related work processes camera image snippets [3] directly. Due to the advances in human body pose estimation [5,4], related approaches extract the body skeleton data from the images and perform gesture recognition on it [35,22,15]. For processing skeletons to predict the gesture, recurrent neural networks [35] or convolutional neural networks [22] are applied.…”

Section: Related Workmentioning

confidence: 99%

Gesture Recognition with Keypoint and Radar Stream Fusion for Automated Vehicles

Holzbock,

Kern,

Waldschmidt

et al. 2023

Preprint

View full text Add to dashboard Cite

We present a joint camera and radar approach to enable autonomous vehicles to understand and react to human gestures in everyday traffic. Initially, we process the radar data with a PointNet followed by a spatio-temporal multilayer perceptron (stMLP). Independently, the human body pose is extracted from the camera frame and processed with a separate stMLP network. We propose a fusion neural network for both modalities, including an auxiliary loss for each modality. In our experiments with a collected dataset, we show the advantages of gesture recognition with two modalities. Motivated by adverse weather conditions, we also demonstrate promising performance when one of the sensors lacks functionality.

show abstract

“…Furthermore, it is advantageous to incorporate self-supervised learning. Some recent work has already leveraged the intrinsic constraints across multiple third-person views, such as multiview geometry and view consistency [15]- [17], to enable the 3D human pose estimation without ground truth. These self-supervised methods have demonstrated comparative pose estimation performance against those fully-supervised counterparts.…”

Section: Introductionmentioning

confidence: 99%

EgoFish3D: Egocentric 3D Pose Estimation From a Fisheye Camera via Self-Supervised Learning

Liu

Yang

et al. 2023

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Self-Supervised 3D Human Pose Estimation with Multiple-View Geometry

Cited by 20 publications

References 38 publications

PosturePose: Optimized Posture Analysis for Semi-Supervised Monocular 3D Human Pose Estimation

PosturePose: Optimized Posture Analysis for Semi-Supervised Monocular 3D Human Pose Estimation

Gesture Recognition with Keypoint and Radar Stream Fusion for Automated Vehicles

EgoFish3D: Egocentric 3D Pose Estimation From a Fisheye Camera via Self-Supervised Learning

Contact Info

Product

Resources

About