Zero3D: Semantic-Driven Multi-Category 3D Shape Generation

Abstract

Semantic-driven 3D shape generation aims to generate 3D objects conditioned on text. Previous works face problems with single-category generation, low-frequency 3D details, and requiring a large number of paired datasets for training. To tackle these challenges, we propose a multi-category conditional diffusion model. Specifically, 1) to alleviate the problem of lack of large-scale paired data, we bridge the text, 2D image and 3D shape based on the pre-trained CLIP model, and 2) to obtain the multi-category 3D shape feature, we apply the conditional flow model to generate 3D shape vector conditioned on CLIP embedding. 3) to generate multicategory 3D shape, we employ the hidden-layer diffusion model conditioned on the multi-category shape vector, which greatly reduces the training time and memory consumption.

AMD: Autoregressive Motion Diffusion

“…Diffusion-based approach (Tevet et al 2022b;Zhang et al 2022;Xin et al 2023): diffusion models (Ho and Salimans 2022;Song et al 2020) have recently attracted significant attention and have shown remarkable breakthroughs in various areas such as video (Luo et al 2023), image (Ramesh et al 2022), and 3D point cloud generation (Han, Liu, and Shen 2023), etc. Current motion generation methods based on diffusion models (Tevet et al 2022b;Zhang et al 2022;Xin et al 2023) have achieved exceptional results using different denoising strategies.…”