Some Theoretical Insights into Wasserstein GANs

Biau, Gérard; Sangnier, Maxime; Tanielian, Ugo

doi:10.48550/arxiv.2006.02682

Cited by 3 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To the best of our knowledge, this is the first result assessing the influence of the noise and of the contamination on the error of generative modeling. This constitutes an appealing complement to the recently obtained statistical guarantees (Biau et al, 2020b;Luise et al, 2020).…”

Section: Discussionmentioning

confidence: 88%

“…Without any smoothness assumptions, Biau et al (2020a) provide large sample properties of the estimated distribution assuming that all the densities induced by the class of generators are dominated by a fixed known measure on a Borel subset of R D . When the admissible discriminators are neural networks with a given architecture, Biau et al (2020b) obtained the parametric rate n −1/2 . To our knowledge, Luise et al (2020) is the only work which establishes statistical guarantees under the assumption that the data generating process is a smooth transformation of a lowdimensional latent distribution.…”

Section: Related Work (And Contributions)mentioning

confidence: 99%

See 1 more Smart Citation

Statistical guarantees for generative models without domination

Schreuder,

Brunel,

Dalalyan

2020

Preprint

View full text Add to dashboard Cite

In this paper, we introduce a convenient framework for studying (adversarial) generative models from a statistical perspective. It consists in modeling the generative device as a smooth transformation of the unit hypercube of a dimension that is much smaller than that of the ambient space and measuring the quality of the generative model by means of an integral probability metric. In the particular case of integral probability metric defined through a smoothness class, we establish a risk bound quantifying the role of various parameters. In particular, it clearly shows the impact of dimension reduction on the error of the generative model.

show abstract

Section: Discussionmentioning

confidence: 88%

Section: Related Work (And Contributions)mentioning

confidence: 99%

Statistical guarantees for generative models without domination

Schreuder,

Brunel,

Dalalyan

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…The main strand of research on GANs deals with empirical insights and basic mathematical properties. Recently researchers started to analyze the GAN problem from the statistical perspectives [Biau et al, 2020b,a, Liang, 2018, Singh et al, 2018, Luise et al, 2020, Uppal et al, 2019 as well as optimization and algorithmic viewpoints [Liang and Stokes, 2019, Kodali et al, 2017, Pfau and Vinyals, 2016, Nie and Patel, 2020, Nagarajan and Kolter, 2017, Genevay et al, 2018.…”

Section: Introductionmentioning

confidence: 99%

Rates of convergence for density estimation with generative adversarial networks

Denis

Moulines

Naumov

et al. 2021

Preprint

View full text Add to dashboard Cite

We undertake a precise study of the non-asymptotic properties of vanilla generative adversarial networks (GANs) and derive theoretical guarantees in the problem of estimating an unknown d-dimensional density p * under a proper choice of the class of generators and discriminators. We prove that the resulting density estimate converges to p * in terms of Jensen-Shannon (JS) divergence at the rate (log n/n) 2β/(2β+d) where n is the sample size and β determines the smoothness of p * . This is the first result in the literature on density estimation using vanilla GANs with JS rates faster than n −1/2 in the regime β > d/2.

show abstract

“…Further theoretical work on generative adversarial learning that was recently published includes theory on Wasserstein methods [4], as well as theory on domain shifts quantified by means of an adversarial loss that reduces Jensen-Shannon divergence [26].…”

Section: Introductionmentioning

confidence: 99%

A Convenient Infinite Dimensional Framework for Generative Adversarial Learning

Asatryan¹,

Gottschalk²,

Marieke³

et al. 2020

Preprint

View full text Add to dashboard Cite

In recent years, generative adversarial networks (GANs) have demonstrated impressive experimental results while there are only a few works that foster statistical learning theory for GANs. In this work, we propose an infinite dimensional theoretical framework for generative adversarial learning. Assuming the class of uniformly bounded k-times α-Hölder differentiable (C k,α ) and uniformly positive densities, we show that the Rosenblatt transformation induces an optimal generator, which is realizable in the hypothesis space of C k,α generators. With a consistent definition of the hypothesis space of discriminators, we further show that in our framework the Jensen-Shannon (JS) divergence between the distribution induced by the generator from the adversarial learning procedure and the data generating distribution converges to zero. As our convenient framework avoids modeling errors both for generators and discriminators, by the error decomposition for adversarial learning it suffices that the sampling error vanishes in the large sample limit. To prove this, we endow the hypothesis spaces of generators and discriminators with C k,α ′ -topologies, 0 < α ′ < α, which render the hypothesis spaces to compact topological spaces such that the uniform law of large numbers can be applied. Under sufficiently strict regularity assumptions on the density of the data generating process, we also provide rates of convergence based on concentration and chaining. To this avail, we first prove subgaussian properties of the empirical process indexed by generators and discriminators. Furthermore, as covering numbers of bounded sets in C k,α -Hölder spaces with respect to the L ∞ -norm lead to a convergent metric entropy integral if k is sufficiently large, we obtain a finite constant in Dudley's inequality. This, in combination with McDiarmid's inequality, provides explicit rate estimates for the convergence of the GAN learner to the true probability distribution in JS divergence.

show abstract

Some Theoretical Insights into Wasserstein GANs

Cited by 3 publications

References 0 publications

Statistical guarantees for generative models without domination

Statistical guarantees for generative models without domination

Rates of convergence for density estimation with generative adversarial networks

A Convenient Infinite Dimensional Framework for Generative Adversarial Learning

Contact Info

Product

Resources

About