Learning Frequency Domain Priors for Image Demoireing

Abstract

Image demoireing is a multi-faceted image restoration task involving both moire pattern removal and color restoration. In this paper, we raise a general degradation model to describe an image contaminated by moire patterns, and propose a novel multi-scale bandpass convolutional neural network (MBCNN) for single image demoireing. For moire pattern removal, we propose a multi-block-size learnable bandpass filters (M-LBFs), based on a block-wise frequency domain transform, to learn the frequency domain priors of moire patterns. We also introduce a new loss function named Dilated Advanced Sobel loss (D-ASL) to better sense the frequency information. For color restoration, we propose a two-step tone mapping strategy, which first applies a global tone mapping to correct for a global color shift, and then performs local fine tuning of the color per pixel. To determine the most appropriate frequency domain transform, we investigate several transforms including DCT, DFT, DWT, learnable non-linear transform and learnable orthogonal transform. We finally adopt the DCT. Our basic model won the AIM2019 demoireing challenge. Experimental results on three public datasets show that our method outperforms state-of-the-art methods by a large margin.

Image Demoireing via U-Net for Detection of Display Defects

“…Among the methods in the demoireing field, MBCNN [18], a state-of-the-art network, converted the feature of the spatial domain into the feature of the frequency domain through DCT [19] and multiplied the feature by weight to enable weight to be learned. We obtained the localization information and compensated for the lack of information of low-level features by concatenating CBAM, ViT, and the moire pattern removal block (MPRB, MBCNN) [32] in the skip connection.…”

SiamTrans: Zero-Shot Multi-Frame Image Restoration with Pre-trained Siamese Transformers

“…Moiré patterns are mainly caused by the interference between the screen's subpixel layout and the camera's color filter array. Recently, some deep learning models (He et al 2020a;Zheng et al 2020;Yang et al 2020b;He et al 2019;Liu et al 2020a,b;Zheng et al 2021;Yuan et al 2019bYuan et al ,a, 2020 are proposed for single image demoiréing. For multi-frame demoiréing, Liu et al (Liu et al 2020c) use multiple images as inputs and design multi-scale feature encoding modules to enhance low-frequency information.…”

Deep Learning for Screen-Shot Image Demoiréing: A Survey

“…In particular, convolution for downsampling is achieved by changing padding and stride to reduce the image resolution. Convolutional downsampling, which extracts the features of the image and reduces the resolution of the image, is widely used as a downsampling layer in image demoiréing studies [10], [14], [17], [19], [22], [41], [42].…”

“…Subsequently, performing the reshaping operation to produce output size is hs×ws×c. Sub-pixel convolution can solve the checkerboard phenomenon in transposed convolution, and this method is widely used in demoiréing studies [14], [19], [41], [42], [45], [46]. Sub-pixel convolution has a larger perceptual field and provides more contextual information than transposed convolution.…”

“…Yang et al [46] used dense connections to compose information exchange modules that combine features from different branches. Zheng et al [19], Liu et al [31], Zheng et al [41], and Vien et al [45] also employ dense connections in networks, and their dense connection are used between each branch to mitigate gradient disappearance and reduce network parameters.…”