Improved Training of Wasserstein GANs

Gulrajani, Ishaan; Ahmed, Faruk; Arjovsky, Martín; Dumoulin, Vincent; Courville, Aaron

doi:10.48550/arxiv.1704.00028

Cited by 523 publications

(1,010 citation statements)

References 22 publications

Supporting

Mentioning

928

Contrasting

Order By: Relevance

“…We train our denoising diffusion GAN with 4 denoising steps and compare it to other models in Figure 6. We observe that the vanilla GAN suffers severely from mode collapse, and while techniques like WGAN-GP (Gulrajani et al, 2017) improve mode coverage, the sample quality is still limited.…”

Section: Additional Studiesmentioning

confidence: 95%

Tackling the Generative Learning Trilemma with Denoising Diffusion GANs

Xiao¹,

Kreis²,

Vahdat³

2021

Preprint

View full text Add to dashboard Cite

A wide variety of deep generative models has been developed in the past decade. Yet, these models often struggle with simultaneously addressing three key requirements including: high sample quality, mode coverage, and fast sampling. We call the challenge imposed by these requirements the generative learning trilemma, as the existing models often trade some of them for others. Particularly, denoising diffusion models have shown impressive sample quality and diversity, but their expensive sampling does not yet allow them to be applied in many real-world applications. In this paper, we argue that slow sampling in these models is fundamentally attributed to the Gaussian assumption in the denoising step which is justified only for small step sizes. To enable denoising with large steps, and hence, to reduce the total number of denoising steps, we propose to model the denoising distribution using a complex multimodal distribution. We introduce denoising diffusion generative adversarial networks (denoising diffusion GANs) that model each denoising step using a multimodal conditional GAN. Through extensive evaluations, we show that denoising diffusion GANs obtain sample quality and diversity competitive with original diffusion models while being 2000× faster on the CIFAR-10 dataset. Compared to traditional GANs, our model exhibits better mode coverage and sample diversity. To the best of our knowledge, denoising diffusion GAN is the first model that reduces sampling cost in diffusion models to an extent that allows them to be applied to real-world applications inexpensively. Project page and code: https://nvlabs.github.io/denoising-diffusion-gan. * Work done during an internship at NVIDIA.

show abstract

Section: Additional Studiesmentioning

confidence: 95%

Tackling the Generative Learning Trilemma with Denoising Diffusion GANs

Xiao¹,

Kreis²,

Vahdat³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…For the perceptual loss, we calculate it on relu5-1 VGG (Simonyan and Zisserman 2014) features. For the adversarial loss, we employ WGAN-GP (Gulrajani et al 2017). The overall loss function of our model is ultimately designed as:…”

Section: Loss Functionsmentioning

confidence: 99%

Coarse-to-Fine Embedded PatchMatch and Multi-Scale Dynamic Aggregation for Reference-based Super-Resolution

Xia¹,

Tian²,

Hang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Reference-based super-resolution (RefSR) has made significant progress in producing realistic textures using an external reference (Ref) image. However, existing RefSR methods obtain high-quality correspondence matchings consuming quadratic computation resources with respect to the input size, limiting its application. Moreover, these approaches usually suffer from scale misalignments between the lowresolution (LR) image and Ref image. In this paper, we propose an Accelerated Multi-Scale Aggregation network (AMSA) for Reference-based Super-Resolution, including Coarse-to-Fine Embedded PatchMatch (CFE-PatchMatch) and Multi-Scale Dynamic Aggregation (MSDA) module. To improve matching efficiency, we design a novel Embedded PatchMacth scheme with random samples propagation, which involves end-to-end training with asymptotic linear computational cost to the input size. To further reduce computational cost and speed up convergence, we apply the coarseto-fine strategy on Embedded PatchMacth constituting CFE-PatchMatch. To fully leverage reference information across multiple scales and enhance robustness to scale misalignment, we develop the MSDA module consisting of Dynamic Aggregation and Multi-Scale Aggregation. The Dynamic Aggregation corrects minor scale misalignment by dynamically aggregating features, and the Multi-Scale Aggregation brings robustness to large scale misalignment by fusing multi-scale information. Experimental results show that the proposed AMSA achieves superior performance over state-of-the-art approaches on both quantitative and qualitative evaluations.

show abstract

“…The WGAN framework requires the discriminator to be Lipschitz continuous with respect to the input. Therefore, we introduce a gradient penalty term to constrain the gradient of the discriminator [33]:…”

Section: Generative Adversarial Neural Networkmentioning

confidence: 99%

Markov Chain Generative Adversarial Neural Networks for Solving Bayesian Inverse Problems in Physics Applications

Mücke¹,

Sanderse²,

Bohté³

et al. 2021

Preprint

View full text Add to dashboard Cite

In the context of solving inverse problems for physics applications within a Bayesian framework, we present a new approach, Markov Chain Generative Adversarial Neural Networks (MCGANs), to alleviate the computational costs associated with solving the Bayesian inference problem. GANs pose a very suitable framework to aid in the solution of Bayesian inference problems, as they are designed to generate samples from complicated high-dimensional distributions. By training a GAN to sample from a low-dimensional latent space and then embedding it in a Markov Chain Monte Carlo method, we can highly efficiently sample from the posterior, by replacing both the high-dimensional prior and the expensive forward map. We prove that the proposed methodology converges to the true posterior in the Wasserstein-1 distance and that sampling from the latent space is equivalent to sampling in the high-dimensional space in a weak sense. The method is showcased on three test cases where we perform both state and parameter estimation simultaneously. The approach is shown to be up to two orders of magnitude more accurate than alternative approaches while also being up to an order of magnitude computationally faster, in several test cases, including the important engineering setting of detecting leaks in pipelines.

show abstract

Improved Training of Wasserstein GANs

Cited by 523 publications

References 22 publications

Tackling the Generative Learning Trilemma with Denoising Diffusion GANs

Tackling the Generative Learning Trilemma with Denoising Diffusion GANs

Coarse-to-Fine Embedded PatchMatch and Multi-Scale Dynamic Aggregation for Reference-based Super-Resolution

Markov Chain Generative Adversarial Neural Networks for Solving Bayesian Inverse Problems in Physics Applications

Contact Info

Product

Resources

About