Discriminator Feature-Based Inference by Recycling the Discriminator of GANs

Bang, Duhyeon; Kang, Seoungyoon; Shim, Hyunjung

doi:10.1007/s11263-020-01311-4

Cited by 8 publications

(4 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The dual use of

reflects a view of cortical information processing in which several network functions are preferentially shared among a single network mimicking the ventral visual stream ( DiCarlo et al, 2012 ). This approach has been previously successfully employed in machine learning models ( Huang et al, 2018 ; Brock et al, 2017 ; Ulyanov et al, 2017 ; Munjal et al, 2020 ; Bang et al, 2020 ).…”

Section: Resultsmentioning

confidence: 99%

Learning cortical representations through perturbed and adversarial dreaming

Deperrois

Petrovici

Senn

et al. 2022

eLife

View full text Add to dashboard Cite

Humans and other animals learn to extract general concepts from sensory experience without extensive teaching. This ability is thought to be facilitated by offline states like sleep where previous experiences are systemically replayed. However, the characteristic creative nature of dreams suggests that learning semantic representations may go beyond merely replaying previous experiences. We support this hypothesis by implementing a cortical architecture inspired by generative adversarial networks (GANs). Learning in our model is organized across three different global brain states mimicking wakefulness, NREM and REM sleep, optimizing different, but complementary objective functions. We train the model on standard datasets of natural images and evaluate the quality of the learned representations. Our results suggest that generating new, virtual sensory inputs via adversarial dreaming during REM sleep is essential for extracting semantic concepts, while replaying episodic memories via perturbed dreaming during NREM sleep improves the robustness of latent representations. The model provides a new computational perspective on sleep states, memory replay and dreams and suggests a cortical implementation of GANs.

show abstract

“…The dual use of

Section: Resultsmentioning

confidence: 99%

Learning cortical representations through perturbed and adversarial dreaming

Deperrois

Petrovici

Senn

et al. 2022

eLife

View full text Add to dashboard Cite

show abstract

“…Then we can compare images within each bin since they should look alike. For comparison, we use LPIPS [20] which measures the perceptual similarity between semantically similar images, and is more reliable [21] than FID. When comparing images within one bin, we have to consider the case where the numbers of true images and result images that fall to that bin do not match.…”

Section: Motion Transfermentioning

confidence: 99%

Thinking the Fusion Strategy of Multi-reference Face Reenactment

Yashima¹,

Narihira²,

Kojima³

2022

Preprint

View full text Add to dashboard Cite

In recent advances of deep generative models, face reenactment -manipulating and controlling human face, including their head movement-has drawn much attention for its wide range of applicability. Despite its strong expressiveness, it is inevitable that the models fail to reconstruct or accurately generate unseen side of the face of a given single reference image. Most of existing methods alleviate this problem by learning appearances of human faces from large amount of data and generate realistic texture at inference time. Rather than completely relying on what generative models learn, we show that simple extension by using multiple reference images significantly improves generation quality. We show this by 1) conducting the reconstruction task on publicly available dataset, 2) conducting facial motion transfer on our original dataset which consists of multi-person's head movement video sequences, and 3) using a newly proposed evaluation metric to validate that our method achieves better quantitative results.

show abstract

“…This representation is given by the penultimate layer, where its elements are used to estimate the probability to be a true image or cube. This representation in the latent space of the discriminator is semantically meaningful (Bang et al 2020) because it accounts for the presence of specific structures or shapes, and tends to place visually similar images or cubes at a small distance in this space, where they would otherwise be more distant in the space of the images or cubes.…”

Section: Autoencodermentioning

confidence: 99%

Encoding large-scale cosmological structure with generative adversarial networks

Ullmo

Decelle

Aghanim

2021

A&A

View full text Add to dashboard Cite

Recently, a type of neural networks called generative adversarial networks (GANs) has been proposed as a solution for the fast generation of simulation-like datasets in an attempt to avoid intensive computations and running cosmological simulations that are expensive in terms of time and computing power. We built and trained a GAN to determine the strengths and limitations of such an approach in more detail. We then show how we made use of the trained GAN to construct an autoencoder (AE) that can conserve the statistical properties of the data. The GAN and AE were trained on images and cubes issued from two types of N-body simulations, namely 2D and 3D simulations. We find that the GAN successfully generates new images and cubes that are statistically consistent with the data on which it was trained. We then show that the AE can efficiently extract information from simulation data and satisfactorily infers the latent encoding of the GAN to generate data with similar large-scale structures.

show abstract

Discriminator Feature-Based Inference by Recycling the Discriminator of GANs

Cited by 8 publications

References 20 publications

Learning cortical representations through perturbed and adversarial dreaming

Learning cortical representations through perturbed and adversarial dreaming

Thinking the Fusion Strategy of Multi-reference Face Reenactment

Encoding large-scale cosmological structure with generative adversarial networks

Contact Info

Product

Resources

About