A deep learning image inpainting method based on stationary wavelet transform

Abstract

The development of deep learning has greatly improved the image inpainting performance in the past decades. To inpaint images with specific tasks usually require different network models. For instance, the highly structured images need to recover the structural consistency, and the textured ones restore the local high-frequency details. However, it is still challenging to realize an effective algorithm taking account of the global structure and texture details separately. Herein, we proposed a two-stage inpainting method, by combining the information of frequency and spatial domain networks. Stationary wavelet transform (SWT) with good time-frequency characteristics was applied to obtain the sub-band images as the basic input for frequency domain inpainting. Contextual attention layer (CAL) modules were optionally introduced in the network model to adapt to various inpainting tasks. We also tested and discussed the impacts of some commonly used loss functions, including normal L1 loss, normal GAN loss, weighted L1 loss and WGAN-GP loss, on highly structured and textured images.