Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models

Bond-Taylor, Sam; Leach, Adam; Long, Yang; Willcocks, Chris G.

doi:10.1109/tpami.2021.3116668

Cited by 301 publications

(133 citation statements)

References 70 publications

Supporting

Mentioning

102

Contrasting

Order By: Relevance

“…We identified challenges with using normalizing flows for distribution learning, namely, the need for deep networks with many bijective transformations to adequately learn a mapping to the target distribution. For this reason, it is prohibitive from a computational cost standpoint for current flow-based models to be competitive with other architectures on distribution learning tasks [14].…”

Section: Discussionmentioning

confidence: 99%

“…Previous work applying NFs to molecule generation [5] has shown that NFs with post-hoc corrections to enforce chemical validity achieve high validity, novelty, and uniqueness scores on benchmark datasets like QM9 [9] and ZINC250K [10]. However, NFs require deep architectures with many bijective transformations to model complex target distributions [11,12,13], which results in a prohibitive computational cost to training flows [14]. Some applications of generative modeling may call for learning immense, heterogeneous regions of chemical space, which deep DGMs seem well-suited for.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

FastFlows: Flow-Based Models for Molecular Graph Generation

Frey¹,

Gadepally²,

Ramsundar³

2022

Preprint

View full text Add to dashboard Cite

We propose a framework using normalizing-flow based models, SELF-Referencing Embedded Strings, and multi-objective optimization that efficiently generates small molecules. With an initial training set of only 100 small molecules, Fast-Flows generates thousands of chemically valid molecules in seconds. Because of the efficient sampling, substructure filters can be applied as desired to eliminate compounds with unreasonable moieties. Using easily computable and learned metrics for druglikeness, synthetic accessibility, and synthetic complexity, we perform a multi-objective optimization to demonstrate how FastFlows functions in a high-throughput virtual screening context. Our model is significantly simpler and easier to train than autoregressive molecular generative models, and enables fast generation and identification of druglike, synthesizable molecules.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

FastFlows: Flow-Based Models for Molecular Graph Generation

Frey¹,

Gadepally²,

Ramsundar³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Although they can be powerful estimators for the distribution of interest, the training and generation can be exceedingly slow as they are done in a sequential manner ( Table 2 ). 343 A comprehensive benchmark over three different AR-based approaches to mAb generation, conditioned on protein structure can be found in the work of Melnyk et al where “causal convolutions”, GNNs, and transformer-based generation methods are compared. The authors also provide guidelines as to which of the three AR-based approaches are best suited to specific antibody design tasks (e.g., CDR3 grafting, broad or narrow sequence diversity generation).…”

Section: Unconstrained Parameter-driven In Silico Antibody Sequence S...mentioning

confidence: 99%

Progress and challenges for the machine learning-based design of fit-for-purpose monoclonal antibodies

Akbar

Bashour

Rawat

et al. 2022

mAbs

View full text Add to dashboard Cite

Although the therapeutic efficacy and commercial success of monoclonal antibodies (mAbs) are tremendous, the design and discovery of new candidates remain a time and cost-intensive endeavor. In this regard, progress in the generation of data describing antigen binding and developability, computational methodology, and artificial intelligence may pave the way for a new era of in silico on-demand immunotherapeutics design and discovery. Here, we argue that the main necessary machine learning (ML) components for an in silico mAb sequence generator are: understanding of the rules of mAb-antigen binding, capacity to modularly combine mAb design parameters, and algorithms for unconstrained parameter-driven in silico mAb sequence synthesis. We review the current progress toward the realization of these necessary components and discuss the challenges that must be overcome to allow the on-demand ML-based discovery and design of fit-for-purpose mAb therapeutic candidates.

show abstract

“…Common latent variable models in deep learning include energy-based models, variational autoencoders, and flow models [ 28 , 29 ]. The VAE explicitly models the density of the distribution, so it has a prescribed Bayesian Network.…”

Section: Introductionmentioning

confidence: 99%

An Overview of Variational Autoencoders for Source Separation, Finance, and Bio-Signal Applications

Singh

Ogunfunmi

2021

Entropy

View full text Add to dashboard Cite

Autoencoders are a self-supervised learning system where, during training, the output is an approximation of the input. Typically, autoencoders have three parts: Encoder (which produces a compressed latent space representation of the input data), the Latent Space (which retains the knowledge in the input data with reduced dimensionality but preserves maximum information) and the Decoder (which reconstructs the input data from the compressed latent space). Autoencoders have found wide applications in dimensionality reduction, object detection, image classification, and image denoising applications. Variational Autoencoders (VAEs) can be regarded as enhanced Autoencoders where a Bayesian approach is used to learn the probability distribution of the input data. VAEs have found wide applications in generating data for speech, images, and text. In this paper, we present a general comprehensive overview of variational autoencoders. We discuss problems with the VAEs and present several variants of the VAEs that attempt to provide solutions to the problems. We present applications of variational autoencoders for finance (a new and emerging field of application), speech/audio source separation, and biosignal applications. Experimental results are presented for an example of speech source separation to illustrate the powerful application of variants of VAE: VAE, β-VAE, and ITL-AE. We conclude the paper with a summary, and we identify possible areas of research in improving performance of VAEs in particular and deep generative models in general, of which VAEs and generative adversarial networks (GANs) are examples.

show abstract

Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models

Cited by 301 publications

References 70 publications

FastFlows: Flow-Based Models for Molecular Graph Generation

FastFlows: Flow-Based Models for Molecular Graph Generation

Progress and challenges for the machine learning-based design of fit-for-purpose monoclonal antibodies

An Overview of Variational Autoencoders for Source Separation, Finance, and Bio-Signal Applications

Contact Info

Product

Resources

About