One-Shot GAN: Learning to Generate Samples from Single Images and Videos

Sushko, Vadim; Gall, Jürgen; Khoreva, Anna

doi:10.1109/cvprw53098.2021.00293

Cited by 37 publications

(29 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The impressive performance of Sin-GAN proves the feasibility of internal learning on the generation task. From then on, many SIG models (Hinz et al 2021;Chen et al 2021;Zhang, Han, and Guo 2021;Sushko, Gall, and Khoreva 2021;Granot et al 2021) are proposed to explore along this line.…”

Section: Random Synthesesmentioning

confidence: 99%

“…MO-GAN (Chen et al 2021) follows SinGAN but synthesizes the hand-marked regions of interest and the rest of the image separately, and then merges them into an unbroken image. One-shot GAN (Sushko, Gall, and Khoreva 2021) is an endto-end model with multiple discriminators for learning different features of image, but it is not fully convolutional and more like normal GANs with diversity regularization (Yang et al 2019). ExSinGAN (Zhang, Han, and Guo 2021) introduces GAN inversion (Pan et al 2020) and perceptual loss (Johnson, Alahi, and Fei-Fei 2016) into SinGAN to improve performance on non-texture images.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

PetsGAN: Rethinking Priors for Single Image Generation

Zhang

Liu

Han

et al. 2022

AAAI

View full text Add to dashboard Cite

Single image generation (SIG), described as generating diverse samples that have the same visual content as the given natural image, is first introduced by SinGAN, which builds a pyramid of GANs to progressively learn the internal patch distribution of the single image. It shows excellent performance in a wide range of image manipulation tasks. However, SinGAN has some limitations. Firstly, due to lack of semantic information, SinGAN cannot handle the object images well as it does on the scene and texture images. Secondly, the independent progressive training scheme is time-consuming and easy to cause artifacts accumulation. To tackle these problems, in this paper, we dig into the single image generation problem and improve SinGAN by fully-utilization of internal and external priors. The main contributions of this paper include: 1) We interpret single image generation from the perspective of the general generative task, that is, to learn a diverse distribution from the Dirac distribution composed of a single image. In order to solve this non-trivial problem, we construct a regularized latent variable model to formulate SIG. To the best of our knowledge, it is the first time to give a clear formulation and optimization goal of SIG, and all the existing methods for SIG can be regarded as special cases of this model. 2) We design a novel Prior-based end-to-end training GAN (PetsGAN), which is infused with internal prior and external prior to overcome the problems of SinGAN. For one thing, we employ the pre-trained GAN model to inject external prior for image generation, which can alleviate the problem of lack of semantic information and generate natural, reasonable and diverse samples, even for the object image. For another, we fully-utilize the internal prior by a differential Patch Matching module and an effective reconstruction network to generate consistent and realistic texture. 3) We construct abundant of qualitative and quantitative experiments on three datasets. The experimental results show our method surpasses other methods on both generated image quality, diversity, and training speed. Moreover, we apply our method to other image manipulation tasks (e.g., style transfer, harmonization) and the results further prove the effectiveness and efficiency of our method.

show abstract

Section: Random Synthesesmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

PetsGAN: Rethinking Priors for Single Image Generation

Zhang

Liu

Han

et al. 2022

AAAI

View full text Add to dashboard Cite

show abstract

“…Based on SinGAN, ConSinGAN [15] proposes a technique to control the trade-off between fidelity and diversity of generated samples. One-Shot GAN [34] uses a dual-branch discriminator where each head respectively identifies real context and real layout of the generated sample. As one-shot image generation methods focus on exploiting a single image, they are not directly applicable to few-shot image generation tasks where the generator must learn the underlying distribution of a collection of images.…”

Section: Related Workmentioning

confidence: 99%

Smoothing the Generative Latent Space with Mixup-based Distance Learning

Chaerin¹,

Kim²,

Han³

et al. 2021

Preprint

View full text Add to dashboard Cite

Producing diverse and realistic images with generative models such as GANs typically requires large scale training with vast amount of images. GANs trained with extremely limited data can easily overfit to few training samples and display undesirable properties like "stairlike" latent space where transitions in latent space suffer from discontinuity, occasionally yielding abrupt changes in outputs. In this work, we consider the situation where neither large scale dataset of our interest nor transferable source dataset is available, and seek to train existing generative models with minimal overfitting and mode collapse. We propose latent mixup-based distance regularization on the feature space of both a generator and the counterpart discriminator that encourages the two players to reason not only about the scarce observed data points but the relative distances in the feature space they reside. Qualitative and quantitative evaluation on diverse datasets demonstrates that our method is generally applicable to existing models to enhance both fidelity and diversity under the constraint of limited data. Code will be made public.

show abstract

“…Synthetic data generation is also termed data oversampling. Using generative adversarial networks (GANs) as synthetic oversamplers has been a voguish research endeavour for low data regimes [3], [8]. Various researchers have demonstrated that GANs are more effective as compared to other synthetic oversamplers like SMOTE [2], [7], [9], [10].…”

Section: Introductionmentioning

confidence: 99%

Evasion Generative Adversarial Network for Low Data Regimes

Randhawa

Aslam

Alauthman

et al. 2023

IEEE Trans. Artif. Intell.

View full text Add to dashboard Cite

A myriad of recent literary works has leveraged generative adversarial networks (GANs) to generate unseen evasion samples. The purpose is to annex the generated data with the original train set for adversarial training to improve the detection performance of machine learning (ML) classifiers. The quality of generated adversarial samples relies on the adequacy of training data samples. However, in low data regimes like medical diagnostic imaging and cybersecurity, the anomaly samples are scarce in number. This paper proposes a novel GAN design called Evasion Generative Adversarial Network (EVAGAN) that is more suitable for low data regime problems that use oversampling for detection improvement of ML classifiers. EVAGAN not only can generate evasion samples, but its discriminator can act as an evasion-aware classifier. We have considered Auxiliary Classifier GAN (ACGAN) as a benchmark to evaluate the performance of EVAGAN on cybersecurity (ISCX-2014, CIC-2017 and CIC2018) botnet and computer vision (MNIST) datasets. We demonstrate that EVAGAN outperforms ACGAN for unbalanced datasets with respect to detection performance, training stability and time complexity. EVAGAN's generator quickly learns to generate the low sample class and hardens its discriminator simultaneously. In contrast to ML classifiers that require security hardening after being adversarially trained by GAN-generated data, EVAGAN renders it needless. The experimental analysis proves that EVA-GAN is an efficient evasion hardened model for low data regimes for the selected cybersecurity and computer vision datasets. Code will be available at HTTPS://www.github.com/rhr407/EVAGAN. Impact Statement-Artificial Intelligence (AI) applications can help improve the quality of human life. The use of AI is not only limited to medical anomaly detection and drug discovery but can be leveraged in computer networks to keep people safe from malicious activities on the Internet. However, the AI-based models can be biased towards the majority class of data on which they are trained due to data imbalance. Anomaly data samples are always scarce as compared to normal data samples. So this is an open research problem to solve. Our work is an effort to improve the AI-based methods in detection performance, stability and time complexity. Using the proposed technique, we can train our AI model using fewer anomaly samples, improving the costefficiency compared to state-of-the-art in anomaly detection.

show abstract

One-Shot GAN: Learning to Generate Samples from Single Images and Videos

Cited by 37 publications

References 36 publications

PetsGAN: Rethinking Priors for Single Image Generation

PetsGAN: Rethinking Priors for Single Image Generation

Smoothing the Generative Latent Space with Mixup-based Distance Learning

Evasion Generative Adversarial Network for Low Data Regimes

Contact Info

Product

Resources

About