Text‐to‐3D Shape Generation

Abstract

Recent years have seen an explosion of work and interest in text‐to‐3D shape generation. Much of the progress is driven by advances in 3D representations, large‐scale pretraining and representation learning for text and image data enabling generative AI models, and differentiable rendering. Computational systems that can perform text‐to‐3D shape generation have captivated the popular imagination as they enable non‐expert users to easily create 3D content directly from text. However, there are still many limitations and challenges remaining in this problem space. In this state‐of‐the‐art report, we provide a survey of the underlying technology and methods enabling text‐to‐3D shape generation to summarize the background literature. We then derive a systematic categorization of recent work on text‐to‐3D shape generation based on the type of supervision data required. Finally, we discuss limitations of the existing categories of methods, and delineate promising directions for future work.