A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completion

Zhaoyang, Lyu,; Kong, Zhifeng; Xu, Xudong; Peng, Liang; Lin, Dahua

doi:10.48550/arxiv.2112.03530

Cited by 8 publications

(21 citation statements)

References 15 publications

(23 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, diffusion models 23 have received a huge amount of attention in computer vision tasks 242526 , especially in point could generation 272829 which is similar to 3D molecule generation. These methods can inpaint the 3D objects by learning the joint distribution.…”

Section: Mainmentioning

confidence: 99%

A dual diffusion model enables 3D binding bioactive molecule generation and lead optimization given target pockets

Huang

et al. 2023

Preprint

View full text Add to dashboard Cite

Structure-based generative chemistry aims to explore much bigger chemical space to design a ligand with high binding affinity to the target proteins; it is a critical step inde novocomputer-aided drug discovery. Traditionalin silicomethods suffer from calculation inefficiency and the performances of existing machine learning methods could be bottlenecked by the auto-regressive sampling strategy. To address these concerns, we herein have developed a novel conditional deep generative model, PMDM, for 3D molecule generation fitting specified target proteins. PMDM incorporates a dual equivariant diffusion model framework to leverage the local and global molecular dynamics to generate 3D molecules in a one-shot fashion. By considering the conditioned protein semantic information and spatial information, PMDM is able to generate chemically and conformationally valid molecules which suitably fit pocket holes. We have conducted comprehensive experiments to demonstrate that PMDM can generate drug-like, synthesis-accessible, novel, and high-binding affinity molecules targeting specific proteins, outperforming the state-of-the-art (SOTA) models in terms of multiple evaluation metrics. In addition, we perform chemical space analysis for generated molecules and lead compound optimization for SARS-CoV-2 main protease (Mpro) by only utilizing three atoms as the seed fragment. The experimental results implicate that the structures of generated molecules are rational compared to the reference molecules, and PMDM can generate massive bioactive molecules highly binding to the targeted proteins which are not included in the training set.

show abstract

Section: Mainmentioning

confidence: 99%

A dual diffusion model enables 3D binding bioactive molecule generation and lead optimization given target pockets

Huang

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…New metrics, such as BCD [22] and EMD, can be leveraged for evaluation. Recently, diffusion models [9] provide impressive point cloud completion results, which also circumvent the imbalance issue. In addition, unsupervised [31] or selfsupervised [19] point cloud completion can also be studied.…”

Section: Completionmentioning

confidence: 99%

Multi-View Partial (MVP) Point Cloud Challenge 2021 on Completion and Registration: Methods and Results

Peng¹,

Wu²,

Cai³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

As real-scanned point clouds are mostly partial due to occlusions and viewpoints, reconstructing complete 3D shapes based on incomplete observations becomes a fundamental problem for computer vision. With a single incomplete point cloud, it becomes the partial point cloud completion problem. Given multiple different observations, 3D reconstruction can be addressed by performing partialto-partial point cloud registration. Recently, a large-scale Multi-View Partial (MVP) point cloud dataset has been released, which consists of over 100,000 high-quality virtualscanned partial point clouds. Based on the MVP dataset, this paper reports methods and results in the Multi-View Partial Point Cloud Challenge 2021 on Completion and Registration. In total, 128 participants registered for the competition, and 31 teams made valid submissions. The top-ranked solutions will be analyzed, and then we will discuss future research directions.

show abstract

“…Facing these challenges mentioned above, we propose to generate meshes indirectly via an intermediate representation that is easier to model. Inspired by recent successes of deep neural networks in modeling the distribution of point clouds [2,22,24,38,43] and reconstructing meshes from point clouds [11,27], we propose to use point clouds as an intermediate representation of meshes. Consequently, the generation of meshes is effectively reformulated as the generation of point clouds, followed by transforming point clouds into meshes.…”

Section: Introductionmentioning

confidence: 99%

“…Such a reformulation not only enables us to take advantage of the advances of point cloud generation methods, but also successfully bypasses the aforementioned challenges, as the distribution of point clouds is continuous and point clouds are unordered sets without explicit topology. In this paper, we adopt denoising diffusion probabilistic models (DDPMs) [13,32], demonstrated promising results in modeling point clouds [22,24,43], to learn the distribution of the point clouds. And Shape as Points (SAP) [27] is employed to reconstruct meshes from the generated point clouds, which is a powerful surface reconstruction technique that can extract high-quality watertight meshes from point clouds at low inference times.…”

Section: Introductionmentioning

confidence: 99%

Controllable Mesh Generation Through Sparse Latent Point Diffusion Models

Zhaoyang¹,

Wang²,

An³

et al. 2023

Preprint

View full text Add to dashboard Cite

Mesh generation is of great value in various applications involving computer graphics and virtual content, yet designing generative models for meshes is challenging due to their irregular data structure and inconsistent topology of meshes in the same category. In this work, we design a novel sparse latent point diffusion model for mesh generation. Our key insight is to regard point clouds as an intermediate representation of meshes, and model the distribution of point clouds instead. While meshes can be generated from point clouds via techniques like Shape as Points (SAP), the challenges of directly generating meshes can be effectively avoided. To boost the efficiency and controllability of our mesh generation method, we propose to further encode point clouds to a set of sparse latent points with pointwise semantic meaningful features, where two DDPMs are trained in the space of sparse latent points to respectively model the distribution of the latent point positions and features at these latent points. We find that sampling in this latent space is faster than directly sampling dense point clouds. Moreover, the sparse latent points also enable us to explicitly control both the overall structures and local details of the generated meshes. Extensive experiments are conducted on the ShapeNet dataset, where our proposed sparse latent point diffusion model achieves superior performance in terms of generation quality and controllability when compared to existing methods. Project page, code and appendix: https://slide-3d.github.io.

show abstract

A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completion

Cited by 8 publications

References 15 publications

A dual diffusion model enables 3D binding bioactive molecule generation and lead optimization given target pockets

A dual diffusion model enables 3D binding bioactive molecule generation and lead optimization given target pockets

Multi-View Partial (MVP) Point Cloud Challenge 2021 on Completion and Registration: Methods and Results

Controllable Mesh Generation Through Sparse Latent Point Diffusion Models

Contact Info

Product

Resources

About