Relaxed Wasserstein with Applications to GANs

Guo, Xin; Hong, Johnny; Lin, Tianyi; Yang, Nan

doi:10.48550/arxiv.1705.07164

Cited by 9 publications

(10 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Broadly speaking, previous work in GANs study three main properties: (1) Stability where the focus is on the convergence of the commonly used alternating gradient descent approach to global/local optimizers (equilibriums) for GAN's optimization (e.g., [6,[10][11][12][13], etc. ), (2) Formulation where the focus is on designing proper loss functions for GAN's optimization (e.g., WGAN+Weight Clipping [4], WGAN+Gradient Penalty [5], GAN+Spectral Normalization [14], WGAN+Truncated Gradient Penalty [15], relaxed WGAN [16], f -GAN [17], MMD-GAN [18,19] , Least-Squares GAN [20], Boundary equilibrium GAN [21], etc. ), and (3) Generalization where the focus is on understanding the required number of samples to learn a probability model using GANs (e.g., [22]).…”

Section: Prior Workmentioning

confidence: 99%

Understanding GANs: the LQG Setting

Feizi¹,

Farnia²,

Ginart³

et al. 2017

Preprint

View full text Add to dashboard Cite

Generative Adversarial Networks (GANs) have become a popular method to learn a probability model from data. In this paper, we aim to provide an understanding of some of the basic issues surrounding GANs including their formulation, generalization and stability on a simple benchmark where the data has a high-dimensional Gaussian distribution. Even in this simple benchmark, the GAN problem has not been well-understood as we observe that existing state-of-the-art GAN architectures may fail to learn a proper generative distribution owing to (1) stability issues (i.e., convergence to bad local solutions or not converging at all), (2) approximation issues (i.e., having improper global GAN optimizers caused by inappropriate GAN's loss functions), and (3) generalizability issues (i.e., requiring large number of samples for training). In this setup, we propose a GAN architecture which recovers the maximum-likelihood solution and demonstrates fast generalization. Moreover, we analyze global stability of different computational approaches for the proposed GAN optimization and highlight their pros and cons. Finally, we outline an extension of our model-based approach to design GANs in more complex setups than the considered Gaussian benchmark.

show abstract

Section: Prior Workmentioning

confidence: 99%

Understanding GANs: the LQG Setting

Feizi¹,

Farnia²,

Ginart³

et al. 2017

Preprint

View full text Add to dashboard Cite

show abstract

“…Accordingly, the representation (9) to (13) applies to (such kind of) CBD. The corresponding special case of ( 10) is called "a relaxed Wasserstein distance (parameterized by φ) between È and É" in the recent papers of Lin et al [20] and Guo et al [12] for a restrictive setup where È and É are supposed to have compact support; the latter two references do not give connections to divergences of quantile functions, but substantially concentrate on applications to topic sparsity for analyzing usergenerated web content and social media, respectively, to Generative Adversarial Networks (GANs). is quasi-antitone on ]0, ∞[×]0, ∞[ if the generator function φ is strictly convex and thrice continuously differentiable on ]0, ∞[ (and hence, c is obsolete) and the so-called scale connector W is twice continuously differentiable such that -on ]0, ∞[×]0, ∞[ -Υ φ,c,W,W is twice continuously differentiable and ∂ 2 Υ φ,c,W,W (u,v) ∂u∂v ≤ 0 (an explicit formula of the latter is given in the appendix of Kißlinger & Stummer [16]).…”

Section: New Optimal Transport Problemsmentioning

confidence: 99%

Optimal transport with some directed distances

Stummer

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In essence, GANs are minimizing proper divergences between true distribution and the generated distribution: for instance, [21] uses f-divergence, [27] explores scaled Bregman divergence, [1] adopts Wasserstein-1 distance, and [14] proposes relaxed Wasserstein divergence.…”

Section: Review: Gans and Mfgs As Minimax Gamesmentioning

confidence: 99%

Connecting GANs, MFGs, and OT

Cao,

Guo,

Laurière

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Generative Adversarial Networks (GANs), introduced in 2014 [12], have celebrated great empirical success, especially in image generation and processing. Meanwhile, Mean-Field Games (MFGs), established in [17] and [16] as analytically feasible approximations for N -player games, have experienced rapid growth in theoretical studies. In this paper, we establish theoretical connections between GANs and MFGs. Interpreting MFGs as GANs, on one hand, allows us to devise GANs-based algorithm to solve MFGs. Interpreting GANs as MFGs, on the other hand, provides a new and probabilistic foundation for GANs. Moreover, this interpretation helps establish an analytical connection between GANs and Optimal Transport (OT) problems, the connection previously understood mostly from the geometric perspective as in [19].

show abstract

Relaxed Wasserstein with Applications to GANs

Cited by 9 publications

References 16 publications

Understanding GANs: the LQG Setting

Understanding GANs: the LQG Setting

Optimal transport with some directed distances

Connecting GANs, MFGs, and OT

Contact Info

Product

Resources

About