Intraclass Image Augmentation for Defect Detection Using Generative Adversarial Neural Networks

Abstract

Surface defect identification based on computer vision algorithms often leads to inadequate generalization ability due to large intraclass variation. Diversity in lighting conditions, noise components, defect size, shape, and position make the problem challenging. To solve the problem, this paper develops a pixel-level image augmentation method that is based on image-to-image translation with generative adversarial neural networks (GANs) conditioned on fine-grained labels. The GAN model proposed in this work, referred to as Magna-Defect-GAN, is capable of taking control of the image generation process and producing image samples that are highly realistic in terms of variations. Firstly, the surface defect dataset based on the magnetic particle inspection (MPI) method is acquired in a controlled environment. Then, the Magna-Defect-GAN model is trained, and new synthetic image samples with large intraclass variations are generated. These synthetic image samples artificially inflate the training dataset size in terms of intraclass diversity. Finally, the enlarged dataset is used to train a defect identification model. Experimental results demonstrate that the Magna-Defect-GAN model can generate realistic and high-resolution surface defect images up to the resolution of 512 × 512 in a controlled manner. We also show that this augmentation method can boost accuracy and be easily adapted to any other surface defect identification models.

Generating Defective Epoxy Drop Images for Die Attachment in Integrated Circuit Manufacturing via Enhanced Loss Function CycleGAN

“…A review of medical image augmentation papers using a GAN was covered in [ 9 ]. A pixel-level image augmentation technique was developed in [ 10 ] based on image-to-image translation with a GAN. It was trained on a surface defect dataset of magnetic particle images to generate synthesized image samples.…”

“…The UQI compares generated synthesized and real images in terms of luminance, contrast, and structure, reflecting the characteristics of the human visual system. It corresponds to the special case of the SSIM when C 1 = C 2 = 0 in Equation (10) and can be written as the product of three components of correlation, luminance distortion, and contrast distortion, as follows:…”

Enhancing titanium spacer defect detection through reinforcement learning-optimized digital twin and synthetic data generation

“…In Ref. 14, an approach is introduced to address data scarcity in defect detection by proposing a pixel-level data augmentation method. Guided by a controllable vector within a generative adversarial network (GAN) framework, the mask-to-image translation model synthesizes diverse defect images, enhancing intraclass variety.…”

Thermographic augmentation of defects on thermal images using multi-modal data

“…Another study investigated data augmentation for detecting surface defects in a magnetic particle inspection (MPI). 10 They proposed a GAN model named Magna-Defect-GAN to generate synthetic images. The model is trained on an acquired dataset and later used to enlarge the sample size with significant intra-class variations.…”

“…9 In addition to the issue with data scarcity in thermal defect detection, diversity in noise components, defect geometric properties, position, and environmental conditions make the generalization of models more challenging. 10 One of the approaches for addressing data scarcity in thermal defect detection is to generate a sufficient amount of data by augmenting synthetic defects on thermal images. Combining real and synthetic images can help the model converge more effectively.…”