3D Shape Generation and Completion through Point-Voxel Diffusion

Zhou, Linqi; Du, Yilun

doi:10.48550/arxiv.2104.03670

Cited by 6 publications

(7 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Diffusion-based generative models Our work lays a theoretical foundation for , which recognizes that conditional denoising score matching (Song & Ermon, 2019 and discrete-time diffusion-based generative models (Sohl-Dickstein et al, 2015;Ho et al, 2020) can be viewed as learning to revert an inference process (using the plug-in reverse SDE). This line of work has been successfully applied to modeling high dimensional natural images (Dhariwal & Nichol, 2021;Saharia et al, 2021), audio (Kong et al, 2020), 3D point cloud (Cai et al, 2020;Zhou et al, 2021), and discrete data (Hoogeboom et al, 2021).…”

Section: Related Workmentioning

confidence: 99%

A Variational Perspective on Diffusion-Based Generative Models and Score Matching

Huang¹,

Lim²,

Courville³

2021

Preprint

View full text Add to dashboard Cite

Discrete-time diffusion-based generative models and score matching methods have shown promising results in modeling high-dimensional image data. Recently, Song et al. (2021) show that diffusion processes that transform data into noise can be reversed via learning the score function, i.e. the gradient of the logdensity of the perturbed data. They propose to plug the learned score function into an inverse formula to define a generative diffusion process. Despite the empirical success, a theoretical underpinning of this procedure is still lacking. In this work, we approach the (continuous-time) generative diffusion directly and derive a variational framework for likelihood estimation, which includes continuous-time normalizing flows as a special case, and can be seen as an infinitely deep variational autoencoder. Under this framework, we show that minimizing the score-matching loss is equivalent to maximizing a lower bound of the likelihood of the plug-in reverse SDE proposed by , bridging the theoretical gap.Preprint. Under review.

show abstract

Section: Related Workmentioning

confidence: 99%

A Variational Perspective on Diffusion-Based Generative Models and Score Matching

Huang¹,

Lim²,

Courville³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We utilize free-form deformation (FFD) (Sederberg & Parry, 1986) and radial basis function (RBF)-based deformation (Forti & Rozza, 2014) for non-linear transformations. Such deformations are also common in VR/AR games and point clouds from generative models (GAN) (Li et al, 2018a;Zhou et al, 2021). Specifically, we use multi quadratic (ϕ(x) = √ x 2 + r 2 ) and inverse multi quadratic splines (ϕ(x) = (x 2 + r 2 ) − 1 2 ) as the representative RBFs to cover a wide range of deformation types.…”

Section: Transformation Corruptions Patternsmentioning

confidence: 99%

Benchmarking Robustness of 3D Point Cloud Recognition Against Common Corruptions

Sun¹,

Zhang²,

Kailkhura³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep neural networks on 3D point cloud data have been widely used in the real world, especially in safety-critical applications. However, their robustness against corruptions is less studied. In this paper, we present ModelNet40-C, the first comprehensive benchmark on 3D point cloud corruption robustness, consisting of 15 common and realistic corruptions. Our evaluation shows a significant gap between the performances on ModelNet40 and ModelNet40-C for state-of-the-art (SOTA) models. To reduce the gap, we propose a simple but effective method by combining PointCutMix-R and TENT after evaluating a wide range of augmentation and testtime adaptation strategies. We identify a number of critical insights for future studies on corruption robustness in point cloud recognition. For instance, we unveil that Transformer-based architectures with proper training recipes achieve the strongest robustness. We hope our in-depth analysis will motivate the development of robust training strategies or architecture designs in the 3D point cloud domain. Our codebase and dataset are included in https://github. com/jiachens/ModelNet40-C.

show abstract

“…We show results under trajectories of different number of timesteps K. We select the minimum K such that analytic-DPM can outperform the baselines with full timesteps and underline the corresponding results. (Chen et al, 2020;Kong et al, 2020;Popov et al, 2021;Lam et al, 2021), controllable generation (Choi et al, 2021;Sinha et al, 2021), image super-resolution (Saharia et al, 2021;, image-to-image translation (Sasaki et al, 2021), shape generation (Zhou et al, 2021) and time series forecasting (Rasul et al, 2021).…”

Section: Related Workmentioning

confidence: 99%

Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models

Bao¹,

Li²,

Zhu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Diffusion probabilistic models (DPMs) represent a class of powerful generative models. Despite their success, the inference of DPMs is expensive since it generally needs to iterate over thousands of timesteps. A key problem in the inference is to estimate the variance in each timestep of the reverse process. In this work, we present a surprising result that both the optimal reverse variance and the corresponding optimal KL divergence of a DPM have analytic forms w.r.t. its score function. Building upon it, we propose Analytic-DPM, a training-free inference framework that estimates the analytic forms of the variance and KL divergence using the Monte Carlo method and a pretrained score-based model. Further, to correct the potential bias caused by the score-based model, we derive both lower and upper bounds of the optimal variance and clip the estimate for a better result. Empirically, our analytic-DPM improves the log-likelihood of various DPMs, produces high-quality samples, and meanwhile enjoys a 20× to 80× speed up.

show abstract

3D Shape Generation and Completion through Point-Voxel Diffusion

Cited by 6 publications

References 28 publications

A Variational Perspective on Diffusion-Based Generative Models and Score Matching

A Variational Perspective on Diffusion-Based Generative Models and Score Matching

Benchmarking Robustness of 3D Point Cloud Recognition Against Common Corruptions

Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models

Contact Info

Product

Resources

About