Learning Moiré Pattern Elimination in Both Frequency and Spatial Domains for Image Demoiréing

Abstract

Recently, with the rapid development of mobile sensing technology, capturing scene information by mobile sensing devices in the form of images or videos has become a prevalent recording method. However, the moiré pattern phenomenon may occur when the scene contains digital screens or regular strips, which greatly degrade the visual performance and image quality. In this paper, considering the complexity and diversity of moiré patterns, we propose a novel end-to-end image demoiré method, which can learn moiré pattern elimination in both the frequency and spatial domains. To be specific, in the frequency domain, considering the signal energy of moiré pattern is widely distributed in the frequency, we introduce a wavelet transform to decompose the multi-scale image features, which can help the model identify the moiré features more precisely to suppress them effectively. On the other hand, we also design a spatial domain demoiré block (SDDB). The SDDB module can extract moiré features from the mixed features, then subtract them to obtain clean image features. The combination of the frequency domain and the spatial domain enhances the model’s ability in terms of moiré feature recognition and elimination. Finally, extensive experiments demonstrate the superior performance of our proposed method to other state-of-the-art methods. The Grad-CAM results in our ablation study fully indicate the effectiveness of the two proposed blocks in our method.

A new simulation method for moiré control of LED background screen in film and television shooting

“…2,3 Existing studies on demoiré ing can be classified into two approaches: the pre-processing and the post-processing. The former involves the utilization of various optical filters [2][3][4] , while the latter encompasses a series of digital image processing methods 5 that have been enriched by the advancements in deep learning-based algorithms, enabling effective elimination of moiré patterns [6][7][8][9][10][11] . But for the high technical specification images, these methods have limitations as they may result in information loss or increased post-processing costs.…”

A Short Video Classification Framework Based on Cross-Modal Fusion

“…In addition, there is a significant difference between video content features and hashtag text features. It is difficult to match accurate hashtags to meet users' content consumption needs due to insufficient video text information in the method of searching for tags with the same text in videos [15,16]. Moreover, some videos usually do not contain classification information, and video feature analysis is mainly based on understanding visual image information, but lacks text semantic mining, resulting in an underutilization of semantic information [17,18].…”