Bodyfitr: Robust Automatic 3D Human Body Fitting

Saint, Alexandre; Shabayek, Abd El Rahman; Cherenkova, Kseniya; Gusev, Gleb; Aouada, Djamila; Ottersten, Björn

doi:10.1109/icip.2019.8803819

“…For this we are given around 1000 detailed ground truth meshes, S i , obtained by fitting a human body model to human data in minimal clothing from 3DBodyTex.v2. See [13,14] for methods on automatic and robust model fitting proposed by the dataset authors. On these ground truth meshes a scanning process of the Shapify booth is simulated in software.…”

Section: Challengementioning

confidence: 99%

Implicit Feature Networks for Texture Completion from Partial 3D Data

Chibane¹,

Pons-Moll²

2020

Preprint

0

View full text Add to dashboard Cite

Prior work to infer 3D texture use either texture atlases, which require uv-mappings and hence have discontinuities, or colored voxels, which are memory inefficient and limited in resolution. Recent work, predicts RGB color at every XYZ coordinate forming a texture field, but focus on completing texture given a single 2D image. Instead, we focus on 3D texture and geometry completion from partial and incomplete 3D scans. IF-Nets [2] have recently achieved state-of-the-art results on 3D geometry completion using a multi-scale deep feature encoding, but the outputs lack texture. In this work, we generalize IF-Nets to texture completion from partial textured scans of humans and arbitrary objects. Our key insight is that 3D texture completion benefits from incorporating local and global deep features extracted from both the 3D partial texture and completed geometry. Specifically, given the partial 3D texture and the 3D geometry completed with IF-Nets, our model successfully in-paints the missing texture parts in consistence with the completed geometry. Our model won the SHARP ECCV'20 challenge, achieving highest performance on all challenges.

show abstract

“…Anguelov et al [1] learn a parametric model of the human body pose and shape to regularise the completion of a full 3D body shape while handling the large deformations caused by variation in pose. This sort of methods usually requires manual initialisation, as shown by Saint et al [24,22,23], who propose fully-automatic body model fitting approaches by exploiting the colour or texture information in human scans, as available in 3DBodyTex [22]. This allows recovering shapes with relatively large proportions of missing data.…”

Section: Introductionmentioning

confidence: 99%

“…However, body models smooth out the shape details and are limited to the body shape without clothing. In these works, the completion of the texture is not considered, even though the texture might be used to regularise the shape completion [22,23]. Some works use volumetric convolutions to complete partial 3D shapes [28,3,10].…”

Section: Introductionmentioning

confidence: 99%

3DBooSTeR: 3D Body Shape and Texture Recovery

Saint

¹

,

Kacem

²

,

Cherenkova

³

et al. 2020

Computer Vision – ECCV 2020 Workshops

Self Cite

View full text Add to dashboard Cite

We propose 3DBooSTeR, a novel method to recover a textured 3D body mesh from a textured partial 3D scan. With the advent of virtual and augmented reality, there is a demand for creating realistic and high-fidelity digital 3D human representations. However, 3D scanning systems can only capture the 3D human body shape up to some level of defects due to its complexity, including occlusion between body parts, varying levels of details, shape deformations and the articulated skeleton. Textured 3D mesh completion is thus important to enhance 3D acquisitions. The proposed approach decouples the shape and texture completion into two sequential tasks. The shape is recovered by an encoder-decoder network deforming a template body mesh. The texture is subsequently obtained by projecting the partial texture onto the template mesh before inpainting the corresponding texture map with a novel approach. The approach is validated on the 3DBodyTex.v2 dataset.

show abstract

“…Anguelov et al [1] learn a parametric model of the human body pose and shape to regularise the completion of a full 3D body shape while handling the large deformations caused by variation in pose. This sort of methods usually requires manual intialisation, as shown by Saint et al [21,22], who propose fullyautomatic body model fitting approaches by exploiting the colour or texture information in human scans, as available in 3DBodyTex [21]. This allows recovering shapes with relatively large proportions of missing data.…”

Section: Introductionmentioning

confidence: 99%

“…However, body models smooth out the shape details and are limited to the body shape without clothing. In these works, the completion of the texture is not considered, even though the texture might be used to regularise the shape completion [21,22]. Some works use volumetric convolutions to complete partial 3D shapes [26,3,10].…”

Section: Introductionmentioning

confidence: 99%

3DBooSTeR: 3D Body Shape and Texture Recovery

Saint¹,

Kacem²,

Cherenkova³

et al. 2020

Preprint

Self Cite

0

View full text Add to dashboard Cite

We propose 3DBooSTeR, a novel method to recover a textured 3D body mesh from a textured partial 3D scan. With the advent of virtual and augmented reality, there is a demand for creating realistic and high-fidelity digital 3D human representations. However, 3D scanning systems can only capture the 3D human body shape up to some level of defects due to its complexity, including occlusion between body parts, varying levels of details, shape deformations and the articulated skeleton. Textured 3D mesh completion is thus important to enhance 3D acquisitions. The proposed approach decouples the shape and texture completion into two sequential tasks. The shape is recovered by an encoder-decoder network deforming a template body mesh. The texture is subsequently obtained by projecting the partial texture onto the template mesh before inpainting the corresponding texture map with a novel approach. The approach is validated on the 3DBodyTex.v2 dataset.

show abstract

Bodyfitr: Robust Automatic 3D Human Body Fitting

Cited by 15 publications

References 20 publications

Implicit Feature Networks for Texture Completion from Partial 3D Data

Implicit Feature Networks for Texture Completion from Partial 3D Data

3DBooSTeR: 3D Body Shape and Texture Recovery

3DBooSTeR: 3D Body Shape and Texture Recovery

Contact Info

Product

Resources

About