MGGAN: Solving Mode Collapse Using Manifold-Guided Training

Bang, Duhyeon; Shim, Hyunjung

doi:10.1109/iccvw54120.2021.00266

Cited by 46 publications

(36 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Level 1: Traditional mode collapse [17,19,11,27,26,20,21] -strange, inappropriate patterns become a point attractor in the non-linear cyclic process [28] between the generator and the discriminator (cf. Figure 6 in [17]); Level 2: Light mode collapse [12,29,3,30,31] -patterns from the training set become wholly or partially an attractor for the generator. This is akin to lookup-table behavior made possible by the high number of parameters in a GAN.…”

Section: Avoiding Mode Collapsementioning

confidence: 99%

DiverGAN: An Efficient and Effective Single-Stage Framework for Diverse Text-to-Image Generation

Zhang,

Schomaker

2021

Preprint

View full text Add to dashboard Cite

In this paper, we concentrate on the text-to-image synthesis task that aims at automatically producing perceptually realistic pictures from text descriptions. Recently, several single-stage methods have been proposed to deal with the problems of a more complicated multi-stage modular architecture. However, they often suffer from the lack-of-diversity issue, yielding similar outputs given a single textual sequence. To this end, we present an efficient and effective single-stage framework (DiverGAN) to generate diverse, plausible and semantically consistent images according to a natural-language description. DiverGAN adopts two novel word-level attention modules, i.e., a channel-attention module (CAM) and a pixel-attention module (PAM), which model the importance of each word in the given sentence while allowing the network to assign larger weights to the significant channels and pixels semantically aligning with the salient words. After that, Conditional Adaptive Instance-Layer Normalization (CAdaILN) is introduced to enable the linguistic cues from the sentence embedding to flexibly manipulate the amount of change in shape and texture, further improving visual-semantic representation and helping stabilize the training. Also, a dual-residual structure is developed to preserve more original visual features while allowing for deeper networks, resulting in faster convergence speed and more vivid details. Furthermore, we propose to plug a fully-connected layer into the pipeline to address the lack-of-diversity problem, since we observe that a dense layer will remarkably enhance the generative capability of the network, balancing the trade-off between a low-dimensional random latent code contributing to variants and modulation modules that use high-dimensional and textual contexts to strength feature maps. Inserting a linear layer after the second residual block achieves the best variety and quality. Both qualitative and quantitative results on benchmark data sets demonstrate the superiority of our DiverGAN for realizing diversity, without harming quality and semantic consistency.

show abstract

Section: Avoiding Mode Collapsementioning

confidence: 99%

DiverGAN: An Efficient and Effective Single-Stage Framework for Diverse Text-to-Image Generation

Zhang,

Schomaker

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…This encoder structuring the information is important to "bridge the gap" between generative and discriminative tasks. It is still rare in generative models [23,9,7] and is also a direction to help to prevent mode collapse [38,1]. Unlike approaches like Engel et al [11] that assume the generative model separated the identity and attribute information in the latent space without supervision, we choose to explicitly enforce a structure in the latent space.…”

Section: Related Workmentioning

confidence: 99%

“…Identity source: Image x (1) Attributes source: Image x (2) how much of the "undesired" information has been removed through disentangling 1 . To summarize the overall quality of a model, we propose an aggregated metric that averages those 4 values.…”

Section: Disentangling Evaluationmentioning

confidence: 99%

“…It is then evaluated on all the remaining images of the original dataset. (c) Reconstruction using identity h y from x (1) and attributes h z from x (2) for our model and a baseline that do not use attribute labels for training [31]. We see that our model does use attributes from image 2 while keeping the identity of image 1, whereas the baseline did not manage to remove attribute information from h y and still uses attributes of image 1.…”

Section: C2 Semi-supervised Learningmentioning

confidence: 99%

“…Identity source: Image x (1) Attributes source: Image x attribute removed + attribute added For the semi-supervised experiment, we tried both using a simple semi-supervision by ignoring unlabeled samples when labels were needed, and the more advanced Mean Teacher [42] strategy but found small differences, which might be due to a sub-optimal tuning of the sensitive hyper-parameters of Mean Teacher. We therefore report results for the simple solution in Table 6.…”

Section: C2 Semi-supervised Learningmentioning

confidence: 99%

See 2 more Smart Citations

DualDis: Dual-Branch Disentangling with Adversarial Learning

Robert,

Thome,

Cord

2019

Preprint

View full text Add to dashboard Cite

In computer vision, disentangling techniques aim at improving latent representations of images by modeling factors of variation. In this paper, we propose DualDis, a new auto-encoder-based framework that disentangles and linearizes class and attribute information. This is achieved thanks to a two-branch architecture forcing the separation of the two kinds of information, accompanied by a decoder for image reconstruction and generation. To effectively separate the information, we propose to use a combination of regular and adversarial classifiers to guide the two branches in specializing for class and attribute information respectively. We also investigate the possibility of using semi-supervised learning for an effective disentangling even using few labels. We leverage the linearization property of the latent spaces for semantic image editing and generation of new images. We validate our approach on CelebA, Yale-B and NORB by measuring the efficiency of information separation via classification metrics, visual image manipulation and data augmentation.Preprint. Under review.

show abstract

A comprehensive review of generative adversarial networks: Fundamentals, applications, and challenges

Mohammed

2023

WIREs Computational Stats

View full text Add to dashboard Cite

In machine learning, a generative model is responsible for generating new samples of data in terms of a probabilistic model. Generative adversarial network (GAN) has been widely used to generate realistic samples in different domains and outperforms its peers in the generative models family. However, producing a robust GAN model is not a trivial task because many challenges face the GAN during the training process and impact its performance, affecting the quality and diversity of the generated samples. In this article, we conduct a comprehensive review of GANs to present the fundamentals of GAN, including its components, types, and objective functions. Also, we present an overview of the evaluation matrices used to evaluate GAN models. Moreover, we list the applications of GANs and research work in various domains. Finally, we present the challenges that face GANs and highlight two significant issues, representing mode collapse and training instability, in addition to those research efforts that tackle these challenges.This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Deep Learning Statistical Learning and Exploratory Methods of the Data Sciences > Neural Networks

show abstract

MGGAN: Solving Mode Collapse Using Manifold-Guided Training

Cited by 46 publications

References 18 publications

DiverGAN: An Efficient and Effective Single-Stage Framework for Diverse Text-to-Image Generation

DiverGAN: An Efficient and Effective Single-Stage Framework for Diverse Text-to-Image Generation

DualDis: Dual-Branch Disentangling with Adversarial Learning

A comprehensive review of generative adversarial networks: Fundamentals, applications, and challenges

Contact Info

Product

Resources

About