Unsupervised Domain-Specific Deblurring via Disentangled Representations

Lu, Bao-Liang; Chen, Jun-Cheng; Chellappa, Rama

doi:10.1109/cvpr.2019.01047

Cited by 157 publications

(110 citation statements)

References 43 publications

Supporting

Mentioning

110

Contrasting

Order By: Relevance

“…We compared the performance of our MSPL framework with recent methods based on CNN models [7], [8], [58], [68], [70]. All the experiments were conducted using the official codes provided by the authors [7], [8], [58], [68], [70]. For Xia and Chakrabarti [58], we used the model trained in a supervised manner, as this model has been reported as the best model in their studies.…”

Section: E Comparisons With Existing Methodsmentioning

confidence: 99%

“…The feature distance (d V GG ) of the VGG- Face network [67] was measured to compare the similarity in facial identity between the GT images and the deblurred face images. Following [68], we computed the L 2 distance using the output features from the P ool5 layer of the VGG-Face network [67]. Following the 2020 NTIRE challenge [63], we employed the LPIPS [69] distance, which is computed as the L 2 distance using the output features from the learned CNN for computing human visual perception.…”

Section: Evaluation Metricsmentioning

confidence: 99%

See 1 more Smart Citation

Progressive Semantic Face Deblurring

2020

View full text Add to dashboard Cite

Previous face deblurring methods have utilized semantic segmentation maps as prior knowledge. Most of these methods generated the segmentation map from a blurred facial image, and restore it using the map in a sequential manner. However, the accuracy of the segmentation affects the restoration performance. Generally, it is difficult to obtain an accurate segmentation map from a blurred image. Instead of sequential methods, we propose an efficient method that learns the flows of facial component restoration without performing segmentation. To this end, we propose a multi-semantic progressive learning (MSPL) framework that progressively restores the entire face image starting from the facial components such as the skin, followed by the hair, and the inner parts (eyes, nose, and mouth). Furthermore, we propose a discriminator that observes the reconstruction-flow of the generator. In addition, we present new test datasets to facilitate the comparison of face deblurring methods. Various experiments demonstrate that the proposed MSPL framework achieves higher performance in facial image deblurring compared to the existing methods, both qualitatively and quantitatively. Our code, trained model and data are available at https://github.com/dolphin0104/MSPL-GAN. INDEX TERMS Facial image deblurring, semantic mask, progressive learning, generative adversarial network, deep learning

show abstract

Section: E Comparisons With Existing Methodsmentioning

confidence: 99%

Section: Evaluation Metricsmentioning

confidence: 99%

Progressive Semantic Face Deblurring

2020

View full text Add to dashboard Cite

show abstract

“…Liu et al [24] propose a unified model that learns disentangled representation for describing and manipulating data across multiple domains. For image restoration, Lu et al [25] disentangle the content and blur features from blurred images. Different from [25], we disentangle the content and discriminative representations of multiple degradations.…”

Section: B Representation Disentanglementmentioning

confidence: 99%

“…For image restoration, Lu et al [25] disentangle the content and blur features from blurred images. Different from [25], we disentangle the content and discriminative representations of multiple degradations. We use the content features to restore images and use the discriminative representations for NR-IQA.…”

Section: B Representation Disentanglementmentioning

confidence: 99%

Domain Fingerprints for No-Reference Image Quality Assessment

Xia

Yang

Xue

et al. 2021

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Human fingerprints are detailed and nearly unique markers of human identity. Such a unique and stable fingerprint is also left on each acquired image. It can reveal how an image was degraded during the image acquisition procedure and thus is closely related to the quality of an image. In this work, we propose a new no-reference image quality assessment (NR-IQA) approach called domain-aware IQA (DA-IQA), which for the first time introduces the concept of domain fingerprint to the NR-IQA field. The domain fingerprint of an image is learned from image collections of different degradations and then used as the unique characteristics to identify the degradation sources and assess the quality of the image. To this end, we design a new domain-aware architecture, which enables simultaneous determination of both the distortion sources and the quality of an image. With the distortion in an image better characterized, the image quality can be more accurately assessed, as verified by extensive experiments, which show that the proposed DA-IQA performs better than almost all the compared state-of-the-art NR-IQA methods. Index Terms-No-reference image quality assessment, domain fingerprints, generative adversarial network I. INTRODUCTION N O-reference image quality assessment (NR-IQA) [1-4] is a fundamental yet challenging task that automatically assesses the perceptual quality of a degraded image without the corresponding reference for comparison. It serves as a key component in low-level computer vision, since in many applications it is difficult or even impossible to acquire the non-distorted image as reference to evaluate the quality of a distorted image. The ill-posed nature of NR-IQA is particularly pronounced for the absence of the prior knowledge about distortion form. Numerous efforts [5-12] have been made to extract powerful features to represent images and image degradations. Traditional hand-crafted feature based methods usually leverage natural scene statistics (NSS) [2, 10, 11] and learning-based metrics [7-9]. For example, Saad et al. [11] leverage the statistical features of discrete cosine transform (DCT) for blind image quality assessment. Mittal et al. [2] propose to extract NSS features in the spatial domain to estimate the

show abstract

“…Based on BR-GAN, NR-GAN, and CR-GAN, we further propose blur, noise, and compression robust GAN (BNCR-GAN), which unifies these three models into a single model with additionally introduced adaptive consistency losses that suppress the uncertainty caused by the combination. We provide benchmark scores through large-scale comparative studies on CIFAR-10 and a generality analysis on FFHQ dataset.1 More precisely, to reduce this cost, unpaired learning methods [53,50] were devised.These methods, however, still require separate collection of non-degraded and degraded images. As an alternative, self-supervised learning methods [45, 2, 48] were also proposed; however, their application is limited to denoising.Blur, Noise, and Compression Robust Generative Adversarial Networks 3 Motivated by these existing studies, we address the following problem: "How can we learn a non-degraded image generator directly from degraded images without knowing the details of image degradation? "…”

mentioning

confidence: 99%

Label-Noise Robust Generative Adversarial Networks

Kaneko

Ushiku

Harada

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Recently, generative adversarial networks (GANs), which learn data distributions through adversarial training, have gained special attention owing to their high image reproduction ability. However, one limitation of standard GANs is that they recreate training images faithfully despite image degradation characteristics such as blur, noise, and compression. To remedy this, we address the problem of blur, noise, and compression robust image generation. Our objective is to learn a non-degraded image generator directly from degraded images without prior knowledge of image degradation. The recently proposed noise robust GAN (NR-GAN) already provides a solution to the problem of noise degradation. Therefore, we first focus on blur and compression degradations. We propose blur robust GAN (BR-GAN) and compression robust GAN (CR-GAN), which learn a kernel generator and quality factor generator, respectively, with non-degraded image generators. Owing to the irreversible blur and compression characteristics, adjusting their strengths is non-trivial. Therefore, we incorporate switching architectures that can adapt the strengths in a data-driven manner. Based on BR-GAN, NR-GAN, and CR-GAN, we further propose blur, noise, and compression robust GAN (BNCR-GAN), which unifies these three models into a single model with additionally introduced adaptive consistency losses that suppress the uncertainty caused by the combination. We provide benchmark scores through large-scale comparative studies on CIFAR-10 and a generality analysis on FFHQ dataset.1 More precisely, to reduce this cost, unpaired learning methods [53,50] were devised.These methods, however, still require separate collection of non-degraded and degraded images. As an alternative, self-supervised learning methods [45, 2, 48] were also proposed; however, their application is limited to denoising.Blur, Noise, and Compression Robust Generative Adversarial Networks 3 Motivated by these existing studies, we address the following problem: "How can we learn a non-degraded image generator directly from degraded images without knowing the details of image degradation? " Particularly, in this work, we focus on three representative types of image degradation, i.e., blur, noise, and compression, and call this problem blur, noise, and compression robust image generation. As an example, we demonstrate a solution using our model in Fig. 1(c). Although the same training images ( Fig. 1(a)) are used between baseline GAN (Fig. 1(b)) and proposed blur, noise, and compression robust GAN (BNCR-GAN ) (Fig. 1(c)), BNCR-GAN succeeds in generating non-degraded images, whereas baseline GAN fails to do so.The recently proposed noise robust GAN (NR-GAN) [31] has shown promising results when its application is restricted to noise degradation. Hence, in this work, we first focus on the remaining two kinds of degradation, i.e., blur and compression, and handle all types of degradation as an advanced problem.To obtain robustness in blur and compression degradations, we first propose two variants of ...

show abstract

Unsupervised Domain-Specific Deblurring via Disentangled Representations

Cited by 157 publications

References 43 publications

Progressive Semantic Face Deblurring

Progressive Semantic Face Deblurring

Domain Fingerprints for No-Reference Image Quality Assessment

Label-Noise Robust Generative Adversarial Networks

Contact Info

Product

Resources

About