Deep Head Pose Estimation Using Synthetic Images and Partial Adversarial Domain Adaption for Continuous Label Spaces

Kuhnke, Felix; Ostermann, Jöern

doi:10.1109/iccv.2019.01026

Cited by 48 publications

(54 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For HPE the label space is a continuous distribution, so these proposed methods cannot be applied directly to the HPE problem. The only work that deals with domain adaptation on the regression task, specifically on HPE, is proposed by Kuhnke and Ostermann [42], which reduces the negative transfer from the source outliers through generating source sampler weights during training and propose Partial Adversarial Domain Adaptation for Continuous label spaces (PADACO). This is the only work that trains only on synthetic data rendered from a CG tool and tests on real data.…”

Section: Visual Domain Adaptationmentioning

confidence: 99%

Learning 3D Head Pose From Synthetic Data: A Semi-Supervised Approach

et al. 2021

View full text Add to dashboard Cite

Accurate head pose estimation from 2D image data is an essential component of applications such as driver monitoring systems, virtual reality technology, and human-computer interaction. It enables a better determination of user engagement and attentiveness. The most accurate head pose estimators are based on Deep Neural Networks that are trained with the supervised approach and rely primarily on the accuracy of training data. The acquisition of real head pose data with a wide variation of yaw, pitch and roll is a challenging task. Publicly available head pose datasets have limitations with respect to size, resolution, annotation accuracy and diversity. In this work, a methodology is proposed to generate pixel-perfect synthetic 2D headshot images rendered from high-quality 3D synthetic facial models with accurate head pose annotations. A diverse range of variations in age, race, and gender are also provided. The resulting dataset includes more than 300k pairs of RGB images with corresponding head pose annotations. A wide range of variations in pose, illumination and background are included. The dataset is evaluated by training a state-of-the-art head pose estimation model and testing against the popular evaluation-dataset Biwi. The results show that training with purely synthetic data generated using the proposed methodology achieves close to state-of-the-art results on head pose estimation which are originally trained on real human facial datasets. As there is a domain gap between the synthetic images and real-world images in the feature space, initial experimental results fall short of the current state-of-the-art. To reduce the domain gap, a semisupervised visual domain adaptation approach is proposed, which simultaneously trains with the labelled synthetic data and the unlabeled real data. When domain adaptation is applied, a significant improvement in model performance is achieved. Additionally, by applying a data fusion-based transfer learning approach, better results are achieved than previously published work on this topic.INDEX TERMS Head pose estimation, synthetic face, face dataset, visual domain adaptation.

show abstract

Section: Visual Domain Adaptationmentioning

confidence: 99%

Learning 3D Head Pose From Synthetic Data: A Semi-Supervised Approach

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Outside of the laboratory, [45] proposes an end-to-end approach to estimate zebra pose using a synthetic dataset and jointly estimating a model of the animal pose with a texture map. Another approach is to adversarially train a feature discriminator until the features from the synthetic and real domain are indistinguishable [46,47]. In both humans and animals, we expect that the combination of physical body models and image synthesis will be important for future progress in precise pose estimation.…”

Section: Discussionmentioning

confidence: 99%

WormPose: Image synthesis and convolutional networks for pose estimation in C. elegans

et al. 2021

View full text Add to dashboard Cite

An important model system for understanding genes, neurons and behavior, the nematode worm C. elegans naturally moves through a variety of complex postures, for which estimation from video data is challenging. We introduce an open-source Python package, WormPose, for 2D pose estimation in C. elegans, including self-occluded, coiled shapes. We leverage advances in machine vision afforded from convolutional neural networks and introduce a synthetic yet realistic generative model for images of worm posture, thus avoiding the need for human-labeled training. WormPose is effective and adaptable for imaging conditions across worm tracking efforts. We quantify pose estimation using synthetic data as well as N2 and mutant worms in on-food conditions. We further demonstrate WormPose by analyzing long (∼ 8 hour), fast-sampled (∼ 30 Hz) recordings of on-food N2 worms to provide a posture-scale analysis of roaming/dwelling behaviors.

show abstract

“…Outside of the laboratory, [38] proposes an end-to-end approach to estimate zebra pose using a synthetic dataset and jointly estimating a model of the animal pose with a texture map. Another approach is to adversarially train a feature discriminator until the features from the synthetic and real domain are indistinguishable [39,40]. In both humans and animals, we expect that the combination of physical body models and image synthesis will be important for future progress in precise pose estimation.…”

Section: Discussionmentioning

confidence: 99%

WormPose: Image synthesis and convolutional networks for pose estimation inC. elegans

Hebert¹,

Ahamed²,

Costa

et al. 2020

Preprint

View full text Add to dashboard Cite

An important model system for understanding genes, neurons and behavior, the nematode worm C. elegans naturally moves through a variety of complex postures, for which estimation from video data is challenging. We introduce an open-source Python package, WormPose, for 2D pose estimation in C. elegans , including self-occluded, coiled shapes. We leverage advances in machine vision afforded from convolutional neural networks and introduce a synthetic yet realistic generative model for images of worm posture, thus avoiding the need for human-labeled training. WormPose is effective and adaptable for imaging conditions across worm tracking efforts. We quantify pose estimation using synthetic data as well as N2 and mutant worms in on-food conditions. We further demonstrate WormPose by analyzing long (~10 hour), fast-sampled (~30 Hz) recordings of on-food N2 worms to provide a posture-scale analysis of roaming/dwelling behaviors.

show abstract

Deep Head Pose Estimation Using Synthetic Images and Partial Adversarial Domain Adaption for Continuous Label Spaces

Cited by 48 publications

References 20 publications

Learning 3D Head Pose From Synthetic Data: A Semi-Supervised Approach

Learning 3D Head Pose From Synthetic Data: A Semi-Supervised Approach

WormPose: Image synthesis and convolutional networks for pose estimation in C. elegans

WormPose: Image synthesis and convolutional networks for pose estimation inC. elegans

Contact Info

Product

Resources

About