Automated Sewer Defects Detection Using Style-Based Generative Adversarial Networks and Fine-Tuned Well-Known CNN Classifier

Abstract

Automated sewer defects detection has become an important trend for better management and maintenance of urban sewer systems. Deep learning technology has developed rapidly and offers an innovative solution for automated detection in engineering applications. However, insufficient data and unbalanced samples have proposed a big challenge to deep learning model training. This study adopts the state-of-the-art Style-based Generative Adversarial Networks (StyleGANs) model and compares the performances of its two variants in producing high-quality synthetic sewer defects images. Seven wellknown CNN models are further fine-tuned and trained using the synthetic images for automated sewer defects detection to examine the effects of StyleGANs on augmenting the detection performance. Results show that both StyleGANs are efficient in producing high-quality images with various styles and high-level details for multiple types of sewer defects. Specifically, the StyleGAN2-Adaptive Discriminator Augmentation (StyleGAN2-ADA) with the aid of Freeze Discriminator (Freeze-D) yields the best model performance. Among the adopted CNN classifiers, Inception_v3 achieves the highest detection accuracy. The mean detection accuracy is 94% (with a specific accuracy of 99.7%, 97%, 95.3% and 84% for tree root, residential wall, disjoint and obstacle, respectively) and confirms the reliability of the StyleGANs' performance. The study shows that StyleGANs provide a promising method to alleviate the limited and uneven dataset problem and can improve the deep learning model performance.

Multi-scale hybrid vision transformer and Sinkhorn tokenizer for sewer defect classification

“…Using the Sewer-ML dataset Haurum and Moeslund showed that the sewer defect classification tasks are far from solved, comparing the leading sewer defect classification methods from Kumar et al [18], Meijer et al [19], Xie et al [20], Chen et al [21], Hassan et al [22], and Myrans et al [23]. Concurrent research directions in the sewer defect classification subfield have focused on the usage of StyleGAN-based approaches to increase the effective size of small training dataset [24,25], developing and deploying networks on embedded devices [26,27], and providing defect localization information without explicit localization labels [28,29].…”

“…Recently, the field has also started investigating other parts of the sewer inspection process [30,32,17,[37][38][39][40][41], such as Haurum et al [37] proposing a multi-task classification approach for simultaneously classifying defects, water level, pipe material, and pipe shape, and Wang et al [30] proposed a framework to accurately determine the severity of defects related to the operation and maintenance of the pipes. The field has also adopted recent trends from the general computer vision field such as selfsupervised learning [39], synthetic data generation [25,24,[42][43][44], neural architecture search [45], and usage of the Transformer architecture [17,46], indicating that the automated sewer inspection field is catching up to the general computer vision domain.…”

Enhancing titanium spacer defect detection through reinforcement learning-optimized digital twin and synthetic data generation

“…Recent studies have shown that GANs, especially StyleGAN2, 24 are highly effective in producing synthetic data for defect detection tasks. 25,26 Our research also leverages StyleGAN2 to create synthetic defects superimposed on real background images. These images are then used to train an inspection network.…”

Image Generation and Recognition for Railway Surface Defect Detection

“…The classic GAN model has evolved into various forms and architectures, such as DCGAN [ 13 , 14 , 15 , 16 ], Pix2Pix [ 17 , 18 , 19 ], CycleGAN [ 20 , 21 , 22 ], and StyleGAN [ 23 , 24 , 25 , 26 , 27 , 28 ]. There have been several attempts to generate surface defects using these models.…”

“…Lastly, StyleGAN includes mixing regularization by using two random latent codes to generate a given percentage of images, allowing the model to localize the style to specific image regions. Situ et al [ 26 ] integrated the StyleGAN model with adaptive discriminator augmentation (ADA) to generate synthetic images of surface defects on water sewers. The model was capable of generating high-resolution images.…”