Text2Mesh: Text-Driven Neural Stylization for Meshes

Abstract

Iron ManBrick Lamp Colorful Crochet Candle Astronaut Horse Figure 1. Text2Mesh produces color and geometric details over a variety of source meshes, driven by a target text prompt. Our stylization results coherently blend unique and ostensibly unrelated combinations of text, capturing both global semantics and part-aware attributes.

How to Fill the Optimum Set? Population Gradient Descent with Harmless Diversity

“…We follow the model architecture and generation pipeline in Text2Mesh (Michel et al, 2021). Text2Mesh proposes a neural style field network, which directly outputs the value displacement on the mesh normal and the color on each vertex.…”

“…We first examine and understand our method in some toy examples, and then apply F sum and F max to more difficult deep learning applications: text-to-image (Liu et al, 2021;Ramesh et al, 2021), text-to-mesh (Michel et al, 2021), molecular conformation generation (Shi et al, 2021) and neural network ensemble. In all these cases, we verify and confirm that our method can serve as a plug-in module and can obtain 1) visually more diverse examples, and 2) a better trade-off between main loss (e.g.…”

“…We apply our method to generate diversified meshes of 3D objectives from text. We base our method on Text2Mesh (Michel et al, 2021), and promote the diversity with a CLIP-based semantic diversity score similar to (9) that is based on a pair of test T 1 , T 2 . See Appendix for more detailed setup.…”

MotionCLIP: Exposing Human Motion Generation to CLIP Space

“…Closer to our method are works that utilize the richness of CLIP outside the imagery domain. In the 3D domain, CLIP's latent space provides a useful objective that enables semantic manipulation [Sanghi et al 2021;Michel et al 2021;Wang et al 2021a] where the domain gap is closed by a neural rendering. CLIP is even adopted in temporal domains [Guzhov et al 2021;Luo et al 2021;Fang et al 2021] that utilize large datasets of video sequences that are paired with text and audio.…”

CLIP-Mesh: Generating textured meshes from text using pretrained image-text models

“…Another work generates shapes from text prompts [15] but it requires training of an encoder and decoder using a set of defined meshes which limits generalizability and they also use a voxel representation which lack textures. [7,10] focus on stylization of pre-defined object meshes with text descriptions, while we tackle the problem of generating the entire shape and texture from a detailed natural language description. Concurrent to our work, [6] proposed a zero-shot text guided generation using a NeRF model [11].…”