Equivariant Diffusion for Molecule Generation in 3D

Hoogeboom, Emiel; Satorras, Víctor García; Vignac, Clément; Welling, Max

doi:10.48550/arxiv.2203.17003

Cited by 19 publications

(22 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, while existing models often generate distance matrices [1,25], we instead focus on generating a full set of 3D coordinates, which should improve designability in practice. Our resulting model, ProtDiff, is similar to concurrent work on E(3)-equivariant diffusion models for molecules [17], but with modifications specific to protein structure. Moreover, we develop a novel motif-scaffolding procedure based on Sequential Monte Carlo, SMCDiff, that repurposes an unconditionally trained DPM for conditional sampling.…”

Section: Introductionmentioning

confidence: 92%

“…3D molecule generation. Hoogeboom et al [17] concurrently developed an equivariant diffusion model (EDM) for generating molecules in 3D. EDM generates both 3D coordinates and atomic types.…”

Section: Related Workmentioning

confidence: 99%

“…But it is not clear how to handle conditioning (on the motif) using these approaches. Diffusion probabilistic models (DPMs) offer a potential alternative; not only do they provide a more straightforward path to handling conditioning, but they have also enjoyed success generating small-molecules in 3D [17]. Extending DPMs to Figure 1: Overview of the conditional generative modeling approach to the motif-scaffolding problem.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem

Trippe¹,

Yim²,

Tischer³

et al. 2022

Preprint

View full text Add to dashboard Cite

Construction of a scaffold structure that supports a desired motif, conferring protein function, shows promise for the design of vaccines and enzymes. But a general solution to this motif-scaffolding problem remains open. Current machine-learning techniques for scaffold design are either limited to unrealistically small scaffolds (up to length 20) or struggle to produce multiple diverse scaffolds. We propose to learn a distribution over diverse and longer protein backbone structures via an E(3)equivariant graph neural network. We develop SMCDiff to efficiently sample scaffolds from this distribution conditioned on a given motif; our algorithm is the first to theoretically guarantee conditional samples from a diffusion model in the largecompute limit. We evaluate our designed backbones by how well they align with AlphaFold2-predicted structures. We show that our method can (1) sample scaffolds up to 80 residues and (2) achieve structurally diverse scaffolds for a fixed motif.

show abstract

Section: Introductionmentioning

confidence: 92%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem

Trippe¹,

Yim²,

Tischer³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Diffusion Models for Image Synthesis. Starting with the seminal works of Sohl-Dickstein et al [52] and Ho et al [21], diffusion-based generative models have improved generative modeling of artificial visual systems [11,31,61,23,64,46] and other data [32,24,62] by sequentially removing noise from a random signal to generate an image. Being likelihood-based models, they achieve high data distribution coverage with well-behaved optimization properties while producing high resolution images at unprecedented quality.…”

Section: Related Workmentioning

confidence: 99%

Retrieval-Augmented Diffusion Models

Blattmann¹,

Rombach²,

Oktay³

et al. 2022

Preprint

View full text Add to dashboard Cite

Generative image synthesis with diffusion models has recently achieved excellent visual quality in several tasks such as text-based or class-conditional image synthesis. Much of this success is due to a dramatic increase in the computational capacity invested in training these models. This work presents an alternative approach: inspired by its successful application in natural language processing, we propose to complement the diffusion model with a retrieval-based approach and to introduce an explicit memory in the form of an external database. During training, our diffusion model is trained with similar visual features retrieved via CLIP and from the neighborhood of each training instance. By leveraging CLIP's joint image-text embedding space, our model achieves highly competitive performance on tasks for which it has not been explicitly trained, such as class-conditional or text-image synthesis, and can be conditioned on both text and image embeddings. Moreover, we can apply our approach to unconditional generation, where it achieves state-of-the-art performance. Our approach incurs low computational and memory overheads and is easy to implement. We discuss its relationship to concurrent work and will publish code and pretrained models soon.

show abstract

“…This occurs, for example, in text, segmentation maps, categorical features, discrete latent spaces, and the direct 8-bit representation of images. Previous work has tried to realize the benefits of the denoising framework on discrete data problems, with promising initial results [6,7,8,9,10,11,12,13].…”

Section: Introductionmentioning

confidence: 99%

A Continuous Time Framework for Discrete Denoising Models

Campbell¹,

Joe²,

Bortoli³

et al. 2022

Preprint

View full text Add to dashboard Cite

We provide the first complete continuous time framework for denoising diffusion models of discrete data. This is achieved by formulating the forward noising process and corresponding reverse time generative process as Continuous Time Markov Chains (CTMCs). The model can be efficiently trained using a continuous time version of the ELBO. We simulate the high dimensional CTMC using techniques developed in chemical physics and exploit our continuous time framework to derive high performance samplers that we show can outperform discrete time methods for discrete data. The continuous time treatment also enables us to derive a novel theoretical result bounding the error between the generated sample distribution and the true data distribution.Preprint. Under review.

show abstract

Equivariant Diffusion for Molecule Generation in 3D

Cited by 19 publications

References 23 publications

Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem

Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem

Retrieval-Augmented Diffusion Models

A Continuous Time Framework for Discrete Denoising Models

Contact Info

Product

Resources

About