MGGAN: Solving Mode Collapse using Manifold Guided Training

Bang, Duhyeon; Shim, Hyunjung

doi:10.48550/arxiv.1804.04391

Cited by 8 publications

(12 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We evaluate our regularizer using a simple GAN on a 2D mixture of 8 Gaussians arranged in a circle. This 2D baseline (Metz et al, 2017;Srivastava et al, 2017;Bang & Shim, 2018) provides a simple qualitative evaluation and demonstrates SVMax's potential in unsuper- vised learning. We leverage this simple baseline because we assume b ≥ d, which does not hold for images.…”

Section: Generative Adversarial Networkmentioning

confidence: 94%

SVMax: A Feature Embedding Regularizer

Taha,

Hanson,

Shrivastava

et al. 2021

Preprint

View full text Add to dashboard Cite

A neural network regularizer (e.g., weight decay) boosts performance by explicitly penalizing the complexity of a network. In this paper, we penalize inferior network activations -feature embeddings -which in turn regularize the network's weights implicitly. We propose singular value maximization (SVMax) to learn a more uniform feature embedding. The SVMax regularizer supports both supervised and unsupervised learning. Our formulation mitigates model collapse and enables larger learning rates. We evaluate the SV-Max regularizer using both retrieval and generative adversarial networks. We leverage a synthetic mixture of Gaussians dataset to evaluate SVMax in an unsupervised setting. For retrieval networks, SVMax achieves significant improvement margins across various ranking losses. Code available at https://bit.ly/3jNkgDt

show abstract

Section: Generative Adversarial Networkmentioning

confidence: 94%

SVMax: A Feature Embedding Regularizer

Taha,

Hanson,

Shrivastava

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…For instance, when training on images of digits (MNIST dataset with images for 0 to 9 digits), a GAN model would sometimes generate samples only for a single digit [27]. Recent approaches [28,29,30] tackled mode-collapse in one of two different ways: (1) modifying the learning of the system to reach a better convergence point; or (2) explicitly enforcing the models to capture diverse modes or map back to the true-data distribution. Solutions to the mode-collapse problem range from designing a reconstructor network in VEEGAN [29] to matching the similarity matrix of generated samples with data [31].…”

Section: Problems In Using Gans For Design Synthesismentioning

confidence: 99%

PaDGAN: A Generative Adversarial Network for Performance Augmented Diverse Designs

Chen¹,

Ahmed²

2020

Preprint

View full text Add to dashboard Cite

Deep generative models are proven to be a useful tool for automatic design synthesis and design space exploration. When applied in engineering design, existing generative models face two challenges: 1) generated designs lack diversity and do not cover all areas of the design space and 2) it is difficult to explicitly improve the overall performance or quality of generated designs without excluding low-quality designs from the dataset, which may impair the performance of the trained model due to reduced training sample size. In this paper, we simultaneously address these challenges by proposing a new Determinantal Point Processes based loss function for probabilistic modeling of diversity and quality. With this new loss function, we develop a variant of the Generative Adversarial Network, named "Performance Augmented Diverse Generative Adversarial Network" or PaDGAN, which can generate novel high-quality designs with good coverage of the design space. We demonstrate that PaDGAN can generate diverse and high-quality designs on both synthetic and real-world examples and compare PaDGAN against other models such as the vanilla GAN and the BézierGAN. Unlike typical generative models that usually generate new designs by interpolating within the boundary of training data, we show that PaDGAN expands the design space boundary towards high-quality regions. The proposed method is broadly applicable to many tasks including design space exploration, design optimization, and creative solution recommendation.

show abstract

“…As VEEGAN estimates implicit probability of real data set, it prevents mode collapsing problem and produces more realistic and diverse samples. [3] proposes a manifold guided generative adversarial network (MGGAN) for guiding a generator by adding another adversarial loss in the manifold space transformed by pre-trained encoder network. This enables the generator to learn all modes of target distribution without impairing the quality of an image.…”

Section: Related Workmentioning

confidence: 99%

“…Figure 6 plots HQS and JSD of 2D Grid test of our method at each training iteration with and without mode penalty weights w p and in equation (3). Importance weight achieves around 20% of HQS gain reaching almost 70% within 20 epochs even though it shows fluctuating values in very early iterations.…”

Section: Mixture Of Gaussiansmentioning

confidence: 99%

Mode Penalty Generative Adversarial Network with adapted Auto-encoder

Lee,

Lee

2020

Preprint

View full text Add to dashboard Cite

Generative Adversarial Networks (GAN) are trained to generate sample images of interest distribution. To this end, generator network of GAN learns implicit distribution of real data set from the classification with candidate generated samples. Recently, various GANs have suggested novel ideas for stable optimizing of its networks. However, in real implementation, sometimes they still represent a only narrow part of true distribution or fail to converge. We assume this ill posed problem comes from poor gradient from objective function of discriminator, which easily trap the generator in a bad situation. To address this problem, we propose a mode-penalty GAN combined with pre-trained autoencoder for explicit representation of generated and real data samples in the encoded space. In this space, we make a generator manifold to follow a real manifold by finding entire modes of target distribution. In addition, penalty for uncovered modes of target distribution is given to the generator which encourages it to find overall target distribution. We demonstrate that applying the proposed method to GANs helps generator's optimization becoming more stable and having faster convergence through experimental evaluations. Recently, generative networks have been widely

show abstract

MGGAN: Solving Mode Collapse using Manifold Guided Training

Cited by 8 publications

References 2 publications

SVMax: A Feature Embedding Regularizer

SVMax: A Feature Embedding Regularizer

PaDGAN: A Generative Adversarial Network for Performance Augmented Diverse Designs

Mode Penalty Generative Adversarial Network with adapted Auto-encoder

Contact Info

Product

Resources

About