Generative Adversarial Nets for Robust Scatter Estimation: A Proper Scoring Rule Perspective

Gao, Chao; Yao, Yuan; Zhu, Weizhi

doi:10.48550/arxiv.1903.01944

Cited by 4 publications

(18 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For cost under 2-norm (Theorem E.3), our results generalize the result in Gao et al (2019) from Gaussians to a non-parametric set with a necessary sacrifice of a log factor.…”

Section: Interpretation and Comparisonsupporting

confidence: 70%

“…In Appendix E.5 we discuss optimization aspects of TV H , showing that it can be smoothed to produce a new distance that it closely related to generative adversarial networks (Goodfellow et al, 2014), extending observations in Gao et al (2019) on connections between robustness and GANs.…”

Section: Outline Of Techniquesmentioning

confidence: 94%

“…Finite sample algorithm and analysis. The approach of weakening the distance was mainly studied in the statistics literature (Donoho and Liu, 1988;Chen et al, 2018;Gao et al, , 2019, while the approach of expanding the set was mainly studied in the theoretical computer science literature and led to provably computationally efficient estimators for robust mean/covariance estimation , classification (Klivans et al, 2009;Awasthi et al, 2014;Diakonikolas et al, 2018c), and regression (Klivans et al, 2018;Diakonikolas et al, 2019c). Here we unify the two approaches and provide systematic finite sample analysis for different estimation scenarios, and use the two approaches as a guide to design new algorithms.…”

Section: Design Of Gmentioning

confidence: 99%

“…In the literature on robust mean estimation, G is usually taken to be symmetric or elliptical distributions (Huber, 1973;Donoho and Liu, 1988;Adrover and Yohai, 2002;Gao et al, 2019), Gaussian distributions (Diakonikolas et al, 2016), sub-Gaussian distributions , or distributions with bounded moments (Lai et al, 2016;. recently introduced a condition called resilience that is weaker than the moment type conditions mentioned above and showed it is sufficient to obtain robust recovery guarantees.…”

Section: Introductionmentioning

confidence: 99%

“…We present two approaches for designing MD estimators with good finite-sample rates: weakening the discrepancy and expanding the set of distributions. We also present connections to Gao et al (2019)'s recent analysis of generative adversarial networks for robust estimation.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Generalized Resilience and Robust Statistics

Zhu¹,

Jiao²,

Steinhardt³

2019

Preprint

View full text Add to dashboard Cite

Robust statistics traditionally focuses on outliers, or perturbations in total variation distance. However, a dataset could be corrupted in many other ways, such as systematic measurement errors and missing covariates. We generalize the robust statistics approach to consider perturbations under any Wasserstein distance, and show that robust estimation is possible whenever a distribution's population statistics are robust under a certain family of friendly perturbations. This generalizes a property called resilience previously employed in the special case of mean estimation with outliers. We justify the generalized resilience property by showing that it holds under moment or hypercontractive conditions. Even in the total variation case, these subsume conditions in the literature for mean estimation, regression, and covariance estimation; the resulting analysis simplifies and sometimes improves these known results in both population limit and finite-sample rate. Our robust estimators are based on minimum distance (MD) functionals (Donoho and Liu, 1988), which project onto a set of distributions under a discrepancy related to the perturbation. We present two approaches for designing MD estimators with good finite-sample rates: weakening the discrepancy and expanding the set of distributions. We also present connections to Gao et al. (2019)'s recent analysis of generative adversarial networks for robust estimation.

show abstract

“…For cost under 2-norm (Theorem E.3), our results generalize the result in Gao et al (2019) from Gaussians to a non-parametric set with a necessary sacrifice of a log factor.…”

Section: Interpretation and Comparisonsupporting

confidence: 70%

Section: Outline Of Techniquesmentioning

confidence: 94%

Section: Design Of Gmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Generalized Resilience and Robust Statistics

Zhu¹,

Jiao²,

Steinhardt³

2019

Preprint

View full text Add to dashboard Cite

show abstract

StrokeGAN: Reducing Mode Collapse in Chinese Font Generation via Stroke Encoding

Zeng

Chen

Liu

et al. 2021

AAAI

Self Cite

View full text Add to dashboard Cite

The generation of stylish Chinese fonts is an important problem involved in many applications. Most of existing generation methods are based on the deep generative models, particularly, the generative adversarial networks (GAN) based models. However, these deep generative models may suffer from the mode collapse issue, which significantly degrades the diversity and quality of generated results. In this paper, we introduce a one-bit stroke encoding to capture the key mode information of Chinese characters and then incorporate it into CycleGAN, a popular deep generative model for Chinese font generation. As a result we propose an efficient method called StrokeGAN, mainly motivated by the observation that the stroke encoding contains amount of mode information of Chinese characters. In order to reconstruct the one-bit stroke encoding of the associated generated characters, we introduce a stroke-encoding reconstruction loss imposed on the discriminator. Equipped with such one-bit stroke encoding and stroke-encoding reconstruction loss, the mode collapse issue of CycleGAN can be significantly alleviated, with an improved preservation of strokes and diversity of generated characters. The effectiveness of StrokeGAN is demonstrated by a series of generation tasks over nine datasets with different fonts. The numerical results demonstrate that StrokeGAN generally outperforms the state-of-the-art methods in terms of content and recognition accuracies, as well as certain stroke error, and also generates more realistic characters.

show abstract