GFlowNet Foundations

Bengio, Yoshua; Salem, Lahlou,; Tristan, Deleu,; Tiwari, Mo; Bengio, Emmanuel

doi:10.48550/arxiv.2111.09266

Cited by 14 publications

(22 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent advancements in graph and molecule generation have shown a great potential for designing a wide range of drug candidates with desired properties. Generative Flow Networks (GFlowNets), introduced by [49] as a method to sample a diverse set of candidates in an active learning context, has been found to be capable of generating a diverse set of small molecules [50] and biological sequences such as proteins and DNAs [51]. On the another hand, diffusion models have been popularly adopted to generate the periodic structure of stable materials (e.g.…”

Section: Related Workmentioning

confidence: 99%

Multiresolution equivariant graph variational autoencoder

Son

Kondor

2023

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

In this paper, we propose Multiresolution Equivariant Graph Variational Autoencoders (MGVAE), the first hierarchical generative model to learn and generate graphs in a multiresolution and equivariant manner. At each resolution level, MGVAE employs higher order message passing to encode the graph while learning to partition it into mutually exclusive clusters and coarsening into a lower resolution that eventually creates a hierarchy of latent distributions. MGVAE then constructs a hierarchical generative model to variationally decode into a hierarchy of coarsened graphs. Importantly, our proposed framework is end-to-end permutation equivariant with respect to node ordering. MGVAE achieves competitive results with several generative tasks including general graph generation, molecular generation, unsupervised molecular representation learning to predict molecular properties, link prediction on citation graphs, and graph-based image generation. Our implementation is available at https://github.com/HyTruongSon/MGVAE.

show abstract

Section: Related Workmentioning

confidence: 99%

Multiresolution equivariant graph variational autoencoder

Son

Kondor

2023

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

show abstract

“…We will study the relation between regeneration learning and other iterative-based learning methods such as GFlowNet [4]…”

Section: Related To Regeneration Learningmentioning

confidence: 99%

Regeneration Learning: A Learning Paradigm for Data Generation

Xu¹,

Qin²,

Bian³

et al. 2023

Preprint

View full text Add to dashboard Cite

Machine learning methods for conditional data generation usually build a mapping from source conditional data X to target data Y . The target Y (e.g., text, speech, music, image, video) is usually high-dimensional and complex, and contains information that does not exist in source data, which hinders effective and efficient learning on the source-target mapping. In this paper, we present a learning paradigm called regeneration learning for data generation, which first generates Y (an abstraction/representation of Y ) from X and then generates Y from Y . During training, Y is obtained from Y through either handcrafted rules or selfsupervised learning and is used to learn X → Y and Y → Y . Regeneration learning extends the concept of representation learning to data generation tasks, and can be regarded as a counterpart of traditional representation learning, since 1) regeneration learning handles the abstraction (Y ) of the target data Y for data generation while traditional representation learning handles the abstraction (X ) of source data X for data understanding; 2) both the processes of Y → Y in regeneration learning and X → X in representation learning can be learned in a self-supervised way (e.g., pre-training); 3) both the mappings from X to Y in regeneration learning and from X to Y in representation learning are simpler than the direct mapping from X to Y . We show that regeneration learning can be a widely-used paradigm for data generation (e.g., text generation, speech recognition, speech synthesis, music composition, image generation, and video generation) and can provide valuable insights into developing data generation methods.

show abstract

“…Despite attention variants being the focus of the current study, there are other sub-fields of graph machine learning eyeing different areas of GNNs which deserve equal attention. Other innovative directions in the field of graph machine learning can be attributed to the works of GFlowNets [6,7], the study of how GNNs are aligned with dynamic programming [58], GNNs with combinatorial optimisation [12], Satorras, et al [42]'s Equivariant GNNs, Klicpera et al [31]'s GEMnet, etc.…”

Section: Other Avenues In Graph MLmentioning

confidence: 99%

A Bird's-Eye Tutorial of Graph Attention Architectures

Dhole¹,

Yang²

2022

Preprint

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) have shown tremendous strides in performance for graph-structured problems especially in the domains of natural language processing, computer vision and recommender systems. Inspired by the success of the transformer architecture, there has been an ever-growing body of work on attention variants of GNNs attempting to advance the state of the art in many of these problems. Incorporating "attention" into graph mining has been viewed as a way to overcome the noisiness, heterogenity and complexity associated with graph-structured data as well as to encode soft-inductive bias. It is hence crucial and advantageous to study these variants from a bird's-eye view to assess their strengths and weaknesses. We provide a systematic and focused tutorial centered around attention based GNNs in a hope to benefit researchers dealing with graph-structured problems. Our tutorial looks at GNN variants from the point of view of the attention function and iteratively builds the reader's understanding of different graph attention variants.

show abstract

GFlowNet Foundations

Cited by 14 publications

References 23 publications

Multiresolution equivariant graph variational autoencoder

Multiresolution equivariant graph variational autoencoder

Regeneration Learning: A Learning Paradigm for Data Generation

A Bird's-Eye Tutorial of Graph Attention Architectures

Contact Info

Product

Resources

About