Deep geometric texture synthesis

Abstract

Recently, deep generative adversarial networks for image generation have advanced rapidly; yet, only a small amount of research has focused on generative models for irregular structures, particularly meshes. Nonetheless, mesh generation and synthesis remains a fundamental topic in computer graphics. In this work, we propose a novel framework for synthesizing geometric textures. It learns geometric texture statistics from local neighborhoods (i.e., local triangular patches) of a single reference 3D model. It learns deep features on the faces of the input triangulation, which is used to subdivide and generate offsets across multiple scales, without parameterization of the reference or target mesh. Our network displaces mesh vertices in any direction (i.e., in the normal and tangential direction), enabling synthesis of geometric textures, which cannot be expressed by a simple 2D displacement map. Learning and synthesizing on local geometric patches enables a genus-oblivious framework, facilitating texture transfer between shapes of different genus. CCS Concepts: • Computing methodologies → Neural networks; Shape analysis.

Tm-Net

“…In recent years, neural network based methods have been proposed [Gatys et al 2015;Henzler et al 2021;Snelgrove 2017;Zhou et al 2018] for the synthesis problem, leading to improved results, although these methods are still restricted to image textures, rather than generating textures over surfaces of 3D shapes. Another popular texture synthesis problem is synthesizing geometric textures [Berkiten et al 2017;Hertz et al 2020;Lai et al 2005], which mainly focus on transferring geometric details from an example model to the target surface.…”

Normal‐Driven Spherical Shape Analogies

“…Beyond 2D data, the idea of analogy has also been used for transferring 3D geometric details from one shape to another. We omit the discussion on methods that are not based on analogies, such as mesh cloning [ZHW∗06; TSS∗11] and geometric learning [LKC∗20; HHGC20; WAK∗20; CKF∗21; LZ21], and focus on analogy‐based techniques. Ma et al [MHS∗14] propose a method for 3D style transfer based on patch‐based assembly.…”

HodgeNet

“…Many papers propose algorithms for learning from meshes and other geometric representations. Here, we summarize past approaches for learning features from meshes, although specialized methods for mesh-based learning appear in tasks like generative modeling [Hertz et al 2020;, meshing , and reconstruction .…”