Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One

Grathwohl, Will; Wang, Kuan-Chieh; Jacobsen, Jörn-Henrik; Duvenaud, David; Norouzi, Mohammad; Swersky, Kevin

doi:10.48550/arxiv.1912.03263

Cited by 71 publications

(150 citation statements)

References 22 publications

Supporting

Mentioning

150

Contrasting

Order By: Relevance

“…Energy Based Models. Our work is related to existing work on energy-based models [5,7,9,11,13,23,34,43,46]. Most similar to our work is that of [8], which proposes a framework of utilizing EBMs to compose several object descriptions together.…”

Section: Generated Imagementioning

confidence: 90%

Learning to Compose Visual Relations

Liu¹,

Li²,

Du³

et al. 2021

Preprint

View full text Add to dashboard Cite

The visual world around us can be described as a structured set of objects and their associated relations. An image of a room may be conjured given only the description of the underlying objects and their associated relations. While there has been significant work on designing deep neural networks which may compose individual objects together, less work has been done on composing the individual relations between objects. A principal difficulty is that while the placement of objects is mutually independent, their relations are entangled and dependent on each other. To circumvent this issue, existing works primarily compose relations by utilizing a holistic encoder, in the form of text or graphs. In this work, we instead propose to represent each relation as an unnormalized density (an energy-based model), enabling us to compose separate relations in a factorized manner. We show that such a factorized decomposition allows the model to both generate and edit scenes that have multiple sets of relations more faithfully. We further show that decomposition enables our model to effectively understand the underlying relational scene structure. Project page at: https://composevisualrelations.github.io/ * indicates equal contribution 35th Conference on Neural Information Processing Systems (NeurIPS 2021).

show abstract

Section: Generated Imagementioning

confidence: 90%

Learning to Compose Visual Relations

Liu¹,

Li²,

Du³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Most recent work in energy-based models (EBM) [22] focused on the application of image generative modeling. Neural networks were trained to assign low energy to real samples [12,15,26,31,38] so that realistic-looking samples can be sampled from the low-energy regions of the EBM's energy landscape. Instead of synthesizing new samples, our goal here is to predict RNA splicing outcomes.…”

Section: Energy-based Modelsmentioning

confidence: 99%

“…(Using an uniform discrete distribution as 𝑞 where all 𝑘 possible states (𝑥 𝑖 ) have the same probability 𝑞(𝑥 𝑖 ) = (1/𝑘), we get Eq. (11) and(15), this gives𝑝 𝜃 (𝑥 𝑖 ) = ℎ(𝑥 𝑖 ) 𝑗 ℎ(𝑥 𝑗 ) = exp (−𝐸 𝜃 (𝑥 𝑖 )) 𝑗 exp (−𝐸 𝜃 (𝑥 𝑗 )) = Softmax 𝑖 (𝐸)(16)□…”

mentioning

confidence: 99%

RNA alternative splicing prediction with discrete compositional energy network

Chan

Korsakova

Ong

et al. 2021

Proceedings of the Conference on Health, Inference, and Learning

View full text Add to dashboard Cite

A single gene can encode for different protein versions through a process called alternative splicing. Since proteins play major roles in cellular functions, aberrant splicing profiles can result in a variety of diseases, including cancers. Alternative splicing is determined by the gene's primary sequence and other regulatory factors such as RNA-binding protein levels. With these as input, we formulate the prediction of RNA splicing as a regression task and build a new training dataset (CAPD) to benchmark learned models. We propose discrete compositional energy network (DCEN) which leverages the hierarchical relationships between splice sites, junctions and transcripts to approach this task. In the case of alternative splicing prediction, DCEN models mRNA transcript probabilities through its constituent splice junctions' energy values. These transcript probabilities are subsequently mapped to relative abundance values of key nucleotides and trained with ground-truth experimental measurements. Through our experiments on CAPD 1 , we show that DCEN outperforms baselines and ablation variants. 2 CCS CONCEPTS• Applied computing → Bioinformatics; Health informatics; • Computing methodologies → Neural networks.

show abstract

“…Our work draws on recent work in energy based models (EBMs) [12,14,19,21,34,43,47,53]. Our underlying energy optimization procedure to generate samples is reminiscent of Langevin sampling, which is used to sample from EBMs [12,43,53].…”

Section: Related Workmentioning

confidence: 99%

Unsupervised Learning of Compositional Energy Concepts

Du¹,

Li²,

Sharma³

et al. 2021

Preprint

View full text Add to dashboard Cite

Humans are able to rapidly understand scenes by utilizing concepts extracted from prior experience. Such concepts are diverse, and include global scene descriptors, such as the weather or lighting, as well as local scene descriptors, such as the color or size of a particular object. So far, unsupervised discovery of concepts has focused on either modeling the global scene-level or the local object-level factors of variation, but not both. In this work, we propose COMET, which discovers and represents concepts as separate energy functions, enabling us to represent both global concepts as well as objects under a unified framework. COMET discovers energy functions through recomposing the input image, which we find captures independent factors without additional supervision. Sample generation in COMET is formulated as an optimization process on underlying energy functions, enabling us to generate images with permuted and composed concepts. Finally, discovered visual concepts in COMET generalize well, enabling us to compose concepts between separate modalities of images as well as with other concepts discovered by a separate instance of COMET trained on a different dataset * .

show abstract

Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One

Cited by 71 publications

References 22 publications

Learning to Compose Visual Relations

Learning to Compose Visual Relations

RNA alternative splicing prediction with discrete compositional energy network

Unsupervised Learning of Compositional Energy Concepts

Contact Info

Product

Resources

About