Extracting Triangular 3D Models, Materials, and Lighting From Images

Munkberg, Jacob; Hasselgren, Jon; Shen, Tianchang; Gao, Jun; Chen, Wenzheng; Evans, Alex; Müller, Thomas; Fidler, Sanja

doi:10.1109/cvpr52688.2022.00810

Cited by 210 publications

(74 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One of the key differences between our tetrahedral grid representation and that of DefTet [8] and DMTet [34,44] is how we predict and utilize displacements. In this work, our tetrahedral grid contains a vector-valued displacement to the underlying surface at each tetrahedron's centroid, which we refer to as a deformation field.…”

Section: Related Workmentioning

confidence: 99%

“…While prior works apply the tetrahedral grid representation to the tasks of tetrahedral reconstruction (DefTet) and mesh super-resolution (DMTet), our method provides generative capabilities, a fundamentally different objective. TetGAN enables sampling novel shapes from noise, latent space interpolations, and shape editing, none of which are provided by DefTet/DMTet (nor by inverse rendering [34]). TetGAN achieves this using a novel CNN for tetrahedral meshes, with convolution/pooling blocks that are distinct from DefTet/DMTet components and tailored for the task of generation (inspired by 2D CNNs [23,24,31]).…”

Section: Related Workmentioning

confidence: 99%

“…The irregular tetrahdral grid has recently started to gain traction as an alternative neural representation for 3D geometry [34], pioneered by DefTet and DMTet [8,44]. Using the irregular tetrahedral grid, topological holes can be inserted anywhere easily by removing unoccupied tetrahedra in the grid.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

TetGAN: A Convolutional Neural Network for Tetrahedral Mesh Generation

Gao¹,

Wang²,

Metzer³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present TetGAN, a convolutional neural network designed to generate tetrahedral meshes. We represent shapes using an irregular tetrahedral grid which encodes an occupancy and displacement field. Our formulation enables defining tetrahedral convolution, pooling, and upsampling operations to synthesize explicit mesh connectivity with variable topological genus. The proposed neural network layers learn deep features over each tetrahedron and learn to extract patterns within spatial regions across multiple scales. We illustrate the capabilities of our technique to encode tetrahedral meshes into a semantically meaningful latent-space which can be used for shape editing and synthesis. Our project page is at https://threedle.github.io/tetGAN/.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

TetGAN: A Convolutional Neural Network for Tetrahedral Mesh Generation

Gao¹,

Wang²,

Metzer³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…This post processing flexibility is missing from prior works. More recently, Munkberg et al [38] combine neural and traditional representations within a differentiable rendering framework to recover a triangle mesh, an SVBRDF texture and an environment map, but their approach has only been demonstrated on isolated objects.…”

Section: Learning-based Materials Capturementioning

confidence: 99%

Deep scene-scale material estimation from multi-view indoor captures

Prakash¹,

Rainer²,

Bousseau³

et al. 2022

Computers & Graphics

View full text Add to dashboard Cite

“…Light estimation is one of the sub-tasks of inverse rendering. Most previous works ignore spatially-varying effects and predict a single environment map for the whole scene [Gardner et al 2017;Munkberg et al 2022;Sengupta et al 2019]. Indoor scenes suffer from spatial variations, thus recent work explores spatially-varying lighting estimation for indoor scenes.…”

Section: Related Workmentioning

confidence: 99%