Robust Real-World Image Super-Resolution against Adversarial Attacks

Abstract

Recently deep neural networks (DNNs) have achieved significant success in real-world image super-resolution (SR). However, adversarial image samples with quasi-imperceptible noises could threaten deep learning SR models. In this paper, we propose a robust deep learning framework for real-world SR that randomly erases potential adversarial noises in the frequency domain of input images or features. The rationale is that on the SR task clean images or features have a different pattern from the attacked ones in the frequency domain. Observing that existing adversarial attacks usually add high-frequency noises to input images, we introduce a novel random frequency mask module that blocks out high-frequency components possibly containing the harmful perturbations in a stochastic manner. Since the frequency masking may not only destroys the adversarial perturbations but also affects the sharp details in a clean image, we further develop an adversarial sample classifier based on the frequency domain of images to determine if applying the proposed mask module. Based on the above ideas, we devise a novel real-world image SR framework that combines the proposed frequency mask modules and the proposed adversarial classifier with an existing super-resolution backbone network. Experiments show that our proposed method is more insensitive to adversarial attacks and presents more stable SR results than existing models and defenses. CCS CONCEPTS• Computing methodologies → Image processing.

Kernel Adversarial Learning for Real-world Image Super-resolution

“…Although GANs [Ren et al, 2020, Wang et al, 2018 and the idea of adversarial attacks [Choi et al, 2019, Yue et al, 2021 have been introduced in SR to serve as objective functions for the generation of more photo-realistic images, they were used in a totally different context (from the perspective of model security to defence adversarial attacks) from what we are exploring. Existing work [Kim et al, 2020] adopt down-scaling process of low-resolution image to tackle real-world super-resolution task.…”

‘Pre-training+Fine-tuning’ Model-based 1/2 Folded Image Restoration

“…Recent research has explored the topic of adversarial attacks in low-level image restoration tasks, such as derain [52], superresolution [53]- [55], dehaze [40], deblur [38], and reflection removal [56]. Yu et-al.…”

Cross Domain Low-Dose CT Image Denoising With Semantic Information Alignment

“…Adaptive Version of Random Frequency Mask. Inspired by the random frequency mask module proposed by Yue et al [16], we propose an adaptive version of random frequency mask with latent distribution to extract the semantic information shared by two domains. On the basis of discrete cosine transform (DCT) and inverse-DCT (I-DCT), we mitigate the perturbations encoded as high frequency components in LDCT image.…”

“…Through our preliminary experiments, we find that the binary mask M determined by a Bernoulli distribution with a given probability [16] is not suitable when dealing with LDCT image, as the mask is simply determined by the distance from the lowest-frequency component corresponds to the frequency degree of a component. To efficiently extract the semantic information from LDCT images, learning the latent distributions of semantic-wise is essential.…”