TetraTSDF: 3D Human Reconstruction From a Single Image With a Tetrahedral Outer Shell

Onizuka, Hayato; Hayirci, Zehra; Thomas, Diego; Sugimoto, Akihiro; Uchiyama, Hideaki

doi:10.1109/cvpr42600.2020.00605

Cited by 38 publications

(19 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Onizuka et al [ 26 ] combined a CNN (convolutional neural networks) and PCN (corresponding part connection network) to learn a distribution of the TSDF in the tetrahedral volume from a single image. Huang et al [ 27 ] used parametric 3D human body estimation to construct the semantic space and semantic deformation field, which allows the 2D/3D human body to be converted into a canonical space to reduce geometric blur caused by occlusion in pose changes. Chibane et al [ 30 ] used the 3D multi-scale tensor of deep features for encoding and classified deep features extracted at their location.…”

Section: Related Researchmentioning

confidence: 99%

“…At present, the method that uses a single RGB image as the input is the mainstream, and the ambiguity of the scale of RGB images is an unavoidable limitation. Moreover, using only RGB images to restore the geometric details of the model does not seem to be a reliable method [ 21 , 22 , 23 , 24 , 26 , 27 , 28 ].…”

Section: Related Researchmentioning

confidence: 99%

“…These data-driven methods already encode prior information about the human body, such as posture and body shape. However, the methods based on only RGB input [ 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 ] did not achieve a reliable body shape estimation due to the ambiguity of scale. For another reason, the methods that employed depth input [ 30 , 31 ] have also made some progress, but the recovery of details is unsatisfactory.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Human Motion Tracking with Less Constraint of Initial Posture from a Single RGB-D Sensor

Liu

Wang

et al. 2021

Sensors

View full text Add to dashboard Cite

High-quality and complete human motion 4D reconstruction is of great significance for immersive VR and even human operation. However, it has inevitable self-scanning constraints, and tracking under monocular settings also has strict restrictions. In this paper, we propose a human motion capture system combined with human priors and performance capture that only uses a single RGB-D sensor. To break the self-scanning constraint, we generated a complete mesh only using the front view input to initialize the geometric capture. In order to construct a correct warping field, most previous methods initialize their systems in a strict way. To maintain high fidelity while increasing the easiness of the system, we updated the model while capturing motion. Additionally, we blended in human priors in order to improve the reliability of model warping. Extensive experiments demonstrated that our method can be used more comfortably while maintaining credible geometric warping and remaining free of self-scanning constraints.

show abstract

Section: Related Researchmentioning

confidence: 99%

Section: Related Researchmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Human Motion Tracking with Less Constraint of Initial Posture from a Single RGB-D Sensor

Liu

Wang

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…There were some other approaches in which various cues were used for building sufficient loss function to train the network including the mesh [31], the texture [44], the multi-view images [34], the optimized SMPL model [30] and the video [27,29]. In order to model the detailed appearance, some method attempt to refine the regressed SMPL model to obtain the detailed 3D model [1,3,23,32,42,53,61,62]. In [1,3,32], after estimating the pose and shape of SMPL model, the authors used shape from shading and texture translation to add the details to SMPL like face, hairstyle, and clothes with garment wrinkles.…”

Section: Parametric Human Body Model Based Regressionmentioning

confidence: 99%

Detailed 3D human body reconstruction from multi-view images combining voxel super-resolution and learned implicit representation

Li¹,

Oskarsson²,

Heyden³

2021

Appl Intell

View full text Add to dashboard Cite

The task of reconstructing detailed 3D human body models from images is interesting but challenging in computer vision due to the high freedom of human bodies. This work proposes a coarse-to-fine method to reconstruct detailed 3D human body from multi-view images combining Voxel Super-Resolution (VSR) based on learning the implicit representation. Firstly, the coarse 3D models are estimated by learning an Pixel-aligned Implicit Function based on Multi-scale Features (MF-PIFu) which are extracted by multi-stage hourglass networks from the multi-view images. Then, taking the low resolution voxel grids which are generated by the coarse 3D models as input, the VSR is implemented by learning an implicit function through a multi-stage 3D convolutional neural network. Finally, the refined detailed 3D human body models can be produced by VSR which can preserve the details and reduce the false reconstruction of the coarse 3D models. Benefiting from the implicit representation, the training process in our method is memory efficient and the detailed 3D human body produced by our method from multi-view images is the continuous decision boundary with high-resolution geometry. In addition, the coarse-to-fine method based on MF-PIFu and VSR can remove false reconstructions and preserve the appearance details in the final reconstruction, simultaneously. In the experiments, our method quantitatively and qualitatively achieves the competitive 3D human body models from images with various poses and shapes on both the real and synthetic datasets.

show abstract

“…In the fields of computer vision and graphics, many studies have attempted to reconstruct 3D objects from a single image [ 1 , 2 , 3 , 4 , 5 ]. Techniques of 3D reconstruction from a single image simplify the process and reduce the total computational cost.…”

Section: Introductionmentioning

confidence: 99%

Multiple Sensor Synchronization with theRealSense RGB-D Camera

Yoon

Jang

Huh

et al. 2021

Sensors

View full text Add to dashboard Cite

When reconstructing a 3D object, it is difficult to obtain accurate 3D geometric information using a single camera. In order to capture detailed geometric information of a 3D object, it is inevitable to increase the number of cameras to capture the object. However, cameras need to be synchronized in order to simultaneously capture frames. If cameras are incorrectly synchronized, many artifacts are produced in the reconstructed 3D object. The RealSense RGB-D camera, which is commonly used for obtaining geometric information of a 3D object, provides synchronization modes to mitigate synchronization errors. However, the synchronization modes provided by theRealSense cameras can only sync depth cameras and have limitations in the number of cameras that can be synchronized using a single host due to the hardware issue of stable data transmission. Therefore, in this paper, we propose a novel synchronization method that synchronizes an arbitrary number of RealSense cameras by adjusting the number of hosts to support stable data transmission. Our method establishes a master–slave architecture in order to synchronize the system clocks of the hosts. While synchronizing the system clocks, delays that resulted from the process of synchronization were estimated so that the difference between the system clocks could be minimized. Through synchronization of the system clocks, cameras connected to the different hosts can be synchronized based on the timestamp of the data received by the hosts. Thus, our method synchronizes theRealSense cameras to simultaneously capture accurate 3D information of an object at a constant frame rate without dropping it.

show abstract

TetraTSDF: 3D Human Reconstruction From a Single Image With a Tetrahedral Outer Shell

Cited by 38 publications

References 29 publications

Human Motion Tracking with Less Constraint of Initial Posture from a Single RGB-D Sensor

Human Motion Tracking with Less Constraint of Initial Posture from a Single RGB-D Sensor

Detailed 3D human body reconstruction from multi-view images combining voxel super-resolution and learned implicit representation

Multiple Sensor Synchronization with theRealSense RGB-D Camera

Contact Info

Product

Resources

About