Few-shot Image Generation via Cross-domain Correspondence

Ojha, Utkarsh; Li, Yijun; Lu, Jingwan; Efros, Alexei A.; Lee, Yong Jae; Shechtman, Eli; Zhang, Richard

doi:10.48550/arxiv.2104.06820

Cited by 4 publications

(15 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The transfer-learning works are able to train with extremely few images, as few as none [16]. They typically do so by regularizing training, either with a dedicated loss [30,34] or by restricting which weights are trained [16,33,40]. We note that such works obtain great results on "artistic" domains, such as caricatures.…”

Section: Few-shot Generative Modelsmentioning

confidence: 99%

“…The subsets are of sizes 10, 50, 100 and 200. For each subset, we estimate the diversity by computing the average pairwise LPIPS [34]. Additionally, we tune G p and invert 20 test images following the projection protocol in Section 3.3.…”

Section: Effect Of Dataset Size and Diversitymentioning

confidence: 99%

“…A naive approach to obtain a personalized generative prior, would be to apply a few-shot training method [34,60], on a given set of images of a person. However, as we demonstrate in the paper, these techniques do not adequately preserve the key identity of an individual.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

MyStyle: A Personalized Generative Prior

Nitzan¹,

Aberman²,

He³

et al. 2022

Preprint

View full text Add to dashboard Cite

Figure 1. Using our personalized prior tuned with images of Michelle Obama, we solve various challenging tasks while faithfully preserving her key facial characteristic. Left to right: inpainting, super-resolution, and semantic editing (smile). Each example shows the original input image of Obama, which may be corrupted (top left), and the output based on our personalized face prior (right), compared to a generic face prior (bottom left). The generic face prior is learned from a diverse set of images and produces results that do not preserve Obama's key facial characteristics.

show abstract

Section: Few-shot Generative Modelsmentioning

confidence: 99%

Section: Effect Of Dataset Size and Diversitymentioning

confidence: 99%

See 1 more Smart Citation

MyStyle: A Personalized Generative Prior

Nitzan¹,

Aberman²,

He³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Fine-tuning and Catastrophic Forgetting: Fine-tuning was proven advantageous across fields, settings and tasks and therefore became a standard practice in the deep learning literature. Prominent advantages of fine-tuning are enabling few-shot tasks such as classification and unconditional generation (Wang et al, 2018b;Mo et al, 2020;Wang et al, 2020;Ojha et al, 2021), improved performance in a wide variety of tasks (Devlin et al, 2018;Radford et al, 2018;He et al, 2020) and faster training convergence (Wang et al, 2018b;.…”

Section: Related Workmentioning

confidence: 99%

“…We next experiment with even farther domains, with barely any similarity between parent and child, such as human faces and churches, which were also examined by Ojha et al (2021). Despite lack of commonality, the latent direction that controls face pose in the parent still controls the church pose in the child model (see Figure 14).…”

Section: Analysis Of Aligned Stylegan Modelsmentioning

confidence: 99%

StyleAlign: Analysis and Applications of Aligned StyleGAN Models

Wu¹,

Nitzan²,

Shechtman³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we perform an in-depth study of the properties and applications of aligned generative models. We refer to two models as aligned if they share the same architecture, and one of them (the child) is obtained from the other (the parent) via fine-tuning to another domain, a common practice in transfer learning. Several works already utilize some basic properties of aligned StyleGAN models to perform image-to-image translation. Here, we perform the first detailed exploration of model alignment, also focusing on StyleGAN. First, we empirically analyze aligned models and provide answers to important questions regarding their nature. In particular, we find that the child model's latent spaces are semantically aligned with those of the parent, inheriting incredibly rich semantics, even for distant data domains such as human faces and churches. Second, equipped with this better understanding, we leverage aligned models to solve a diverse set of tasks. In addition to image translation, we demonstrate fully automatic cross-domain image morphing. We further show that zero-shot vision tasks may be performed in the child domain, while relying exclusively on supervision in the parent domain. We demonstrate qualitatively and quantitatively that our approach yields state-of-the-art results, while requiring only simple fine-tuning and inversion.

show abstract

Efficient Defect Classification Using Few-Shot Image Generation and Self-Attention Fused Convolution Features

Zhang,

Yang,

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

Although deep learning has been proven to significantly outperform most traditional methods in the classification of large-scale balanced image datasets, collecting enough samples for defect classification is extremely time-consuming and costly. In this paper, we propose a lightweight defect classification method based on few-shot image generation and self-attention fused convolution features. We constructed a 4-class dataset using welding seam images collected from a solar cell module packaging production line. To address the issue of limited defect samples, especially for classes with less than 10 images, we implemented two strategies. Geometric enhancement techniques were first used to extend the defective images. Secondly, multi-scale feature fusion Generative Adversarial Networks (GANs) were utilized to further enhance the dataset. We then performed the feature-level fusion of convolution neural networks and self-attention networks, achieving a classification accuracy of 98.19%. Our experimental results demonstrate that the proposed model performs well in small sample defect classification tasks. And, it can be effectively applied to product quality inspection tasks in industrial production lines.

show abstract

Few-shot Image Generation via Cross-domain Correspondence

Cited by 4 publications

References 32 publications

MyStyle: A Personalized Generative Prior

MyStyle: A Personalized Generative Prior

StyleAlign: Analysis and Applications of Aligned StyleGAN Models

Efficient Defect Classification Using Few-Shot Image Generation and Self-Attention Fused Convolution Features

Contact Info

Product

Resources

About