Transfer Learning From Synthetic to Real-Noise Denoising With Adaptive Instance Normalization

Kim, Yoon-Sik; Soh, Jae Woong; Park, Gu Yong; Cho, Nam Ik

doi:10.1109/cvpr42600.2020.00354

Cited by 172 publications

(68 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The non-blind denoiser is an U-Net with skip connection, which is used to explore multi-scale features and generate clean images. Later proposed networks inspired by CBDNet achieved higher performance by increasing the network's receptive field [100], [101]. However, to the best of our knowledge, such blind denoising networks were not evaluated in the context of RE and hardware assurance.…”

Section: ) End-to-end Benchmarksmentioning

confidence: 99%

REFICS: Assimilating Data-Driven Paradigms Into Reverse Engineering and Hardware Assurance on Integrated Circuits

Wilson¹,

Zhu

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Comprehensive hardware assurance approaches guaranteeing trust on Integrated Circuits (ICs) typically require the verification of the IC design layout and functionality through destructive Reverse Engineering (RE). It is a resource intensive process that will benefit greatly from the extensive integration of data-driven paradigms, especially in the imaging and image analysis phase. Although obvious, this uptake of data-driven approaches into RE-assisted hardware assurance is lagging due to the lack of massive amounts of high-quality labelled data. In this paper, a large-scale synthetic Scanning Electron Microscopy (SEM) dataset, REFICS, is introduced to address this issue. The dataset, the first open-source dataset in the RE community, consists of 800,000 SEM images over two node technologies, 32nm and 90nm, and four cardinal layers of the IC, namely, doping, polysilicon, contact and metal layers. Furthermore, a framework, based on uncertainty and risk, is introduced to compare the efficacy and benefits of existing RE workflows utilizing ad-hoc steps in its execution. These developments are critical in developing RE-assisted hardware assurance into a scalable, automated and fault-tolerant approach. Finally, the work is concluded with the performance analysis of existing machine learning and deep learning approaches for image analysis in RE and hardware assurance.

show abstract

Section: ) End-to-end Benchmarksmentioning

confidence: 99%

REFICS: Assimilating Data-Driven Paradigms Into Reverse Engineering and Hardware Assurance on Integrated Circuits

Wilson¹,

Zhu

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Generator G c2n For the generator G c2n , we adopt the U-Net based network that is similar to the architecture introduced by [22]. The role of this network is to translate images from the noise domain to the clean domain.…”

Section: B the Details Of Architecturesmentioning

confidence: 99%

Unsupervised Image Denoising with Frequency Domain Knowledge

Kim¹,

Jang²,

Lee³

et al. 2021

Preprint

View full text Add to dashboard Cite

Supervised learning-based methods yield robust denoising results, yet they are inherently limited by the need for large-scale clean/noisy paired datasets. The use of unsupervised denoisers, on the other hand, necessitates a more detailed understanding of the underlying image statistics. In particular, it is well known that apparent differences between clean and noisy images are most prominent on high-frequency bands, justifying the use of low-pass filters as part of conventional image preprocessing steps. However, most learning-based denoising methods utilize only one-sided information from the spatial domain without considering frequency domain information. To address this limitation, in this study we propose a frequency-sensitive unsupervised denoising method. To this end, a generative adversarial network (GAN) is used as a base structure. Subsequently, we include spectral discriminator and frequency reconstruction loss to transfer frequency knowledge into the generator. Results using natural and synthetic datasets indicate that our unsupervised learning method augmented with frequency information achieves stateof-the-art denoising performance, suggesting that frequency domain information could be a viable factor in improving the overall performance of unsupervised learning-based methods.

show abstract

“…With the advancement of transfer learning, Soh et al [31] propose to leverage the Meta-transfer learning to deal with the challenging task of zero-shot super-resolution (ZSSR). Kim et al [17] utilize the adaptive instance normalization to realize the knowledge transfer from synthetic noise to real noise. Moreover, with the development of real image restoration, some real image restoration datasets with limited realworld clean-distorted image pairs have been collected and released [2,38].…”

Section: Image Restorationmentioning

confidence: 99%

“…Mixed mild distortion refers to that the distortion level of Gaussian noise, Gaussian blur and Jpeg artifacts are in the range of [9,11]. Mixed moderate distortion refers to that the distortion level of Gaussian noise, Gaussian blur and Jpeg artifacts are in the range of [12,17], and Mixed severe distortion refers to that the distortion level of Gaussian noise, Gaussian blur and Jpeg artifacts are in the range of [18,20]. More detail about the mixed distortion can be seen in [41].…”

Section: Details Of Auxiliary Distortionsmentioning

confidence: 99%

Few-Shot Real Image Super-resolution via Distortion-Relation Guided Transfer Learning

Li¹,

Jin²,

Fu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Collecting large clean-distorted training image pairs in real world is non-trivial, which seriously limits the practical applications of these supervised learning based image restoration (IR) methods. Previous works attempt to address this problem by leveraging unsupervised learning technologies to alleviate the dependency for paired training samples. However, these methods typically suffer from unsatisfactory textures synthesis due to the lack of clean image supervision. Compared with purely unsupervised solution, the under-explored scheme with Few-Shot clean images (FS-IR) is more feasible to tackle this challenging real Image Restoration task. In this paper, we are the first to investigate the few-shot real image restoration and propose a Distortion-Relation guided Transfer Learning (termed as DRTL) framework. DRTL assigns a knowledge graph to capture the distortion relation between auxiliary tasks (i.e., synthetic distortions) and target tasks (i.e., real distortions with few images), and then adopt a gradient weighting strategy to guide the knowledge transfer from auxiliary task to target task. In this way, DRTL could quickly learn the most relevant knowledge from the prior distortions for target distortion. We instantiate DRTL integrated with pre-training and meta-learning pipelines as an embodiment to realize a distortion-relation aware FS-IR. Extensive experiments on multiple benchmarks demonstrate the effectiveness of DRTL on few-shot real image restoration.

show abstract

Transfer Learning From Synthetic to Real-Noise Denoising With Adaptive Instance Normalization

Cited by 172 publications

References 57 publications

REFICS: Assimilating Data-Driven Paradigms Into Reverse Engineering and Hardware Assurance on Integrated Circuits

REFICS: Assimilating Data-Driven Paradigms Into Reverse Engineering and Hardware Assurance on Integrated Circuits

Unsupervised Image Denoising with Frequency Domain Knowledge

Few-Shot Real Image Super-resolution via Distortion-Relation Guided Transfer Learning

Contact Info

Product

Resources

About