Image Inpainting With Learnable Bidirectional Attention Maps

Xie, Chaohao; Liu, Shaohui; Li, Chao; Cheng, Ming–Ming; Zuo, Wangmeng; Liu, Xiao; Wen, Shifeng; Ding, Errui

doi:10.1109/iccv.2019.00895

Cited by 235 publications

(136 citation statements)

References 31 publications

Supporting

Mentioning

134

Contrasting

Order By: Relevance

“…Nevertheless, we compared our results with those of the latest methods on 2000 test images in CelebA-HQ and 4000 images in the test set of Places2. In the experiments, the irregular mask dataset in Reference [14] were used, and the PSNR and SSIM values were compared, as shown in Tables 5 and 6, where CA [15] represents the generative image inpainting results with Contextual Attention (CVPR2018), PC [14] is the results of Image Inpainting for Irregular Holes Using Partial Convolutions (ECCV2018), EC [17] represents the results of EdgeConnect (ICCV2019), GC [16] represents the results of Free-Form Image Inpainting with Gated Convolution (ICCV2019), LBAM [21] is the results of Learnable Bidirectional Attention Maps (ICCV2019), and RN [23] is the results of Region Normalization for Image Inpainting(AAAI2020). Among them, the data of PC comes from Reference [23,32], while others are performed using the codes or pre-trained models provided by their authors.…”

Section: Methodsmentioning

confidence: 99%

“…Yu et al [16] used the soft mask to instead the binary mask in the network to better represent the restored situation of image. Besides, Xie et al [21] reversed the masks in the encoder network and put them in the decoder network to realize the update of damaged region. In addition, Yang et al [22] considered the structure information in image generation network to produce the realistic structural images.…”

Section: Image Inpainting Based On Deep Learningmentioning

confidence: 99%

“…Besides, EC [17] produces poor contour at stage 1, which further leads to the poor final results. GC [16] and LBAM [21] can get smooth and plausible results; however, some artifacts still exist because the continuity of colors and lines are not well preserved. This is mainly due to the fact that these methods do not pay much attention to the structure information.…”

Section: Qualitative Comparisonsmentioning

confidence: 99%

See 2 more Smart Citations

Image Inpainting Using Two-Stage Loss Function and Global and Local Markovian Discriminators

Liu

et al. 2020

Sensors

View full text Add to dashboard Cite

Image inpainting networks can produce visually reasonable results in the damaged regions. However, existing inpainting networks may fail to reconstruct the proper structures or tend to generate the results with color discrepancy. To solve this issue, this paper proposes an image inpainting approach using the proposed two-stage loss function. The loss function consists of different Gaussian kernels, which are utilized in different stages of network. The use of our two-stage loss function in coarse network helps to focus on the image structure, while the use of it in refinement network is helpful to restore the image details. Moreover, we proposed a global and local PatchGANs (GAN means generative adversarial network), named GL-PatchGANs, in which the global and local markovian discriminators were used to control the final results. This is beneficial to focus on the regions of interest (ROI) on different scales and tends to produce more realistic structural and textural details. We trained our network on three popular datasets on image inpainting separately, both Peak Signal to Noise ratio (PSNR) and Structural Similarity (SSIM) between our results, and ground truths on test images show that our network can achieve better performance compared with the recent works in most cases. Besides, the visual results on three datasets also show that our network can produce visual plausible results compared with the recent works.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Image Inpainting Based On Deep Learningmentioning

confidence: 99%

Section: Qualitative Comparisonsmentioning

confidence: 99%

See 1 more Smart Citation

Image Inpainting Using Two-Stage Loss Function and Global and Local Markovian Discriminators

Liu

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…Wang et al [25] introduce a special multistage attention module that considers structure consistency and detail fineness. To generating fine-grained textures, Xie et al [26] and Yu et al [10] introduce the attention mechanism in image inpainting. Xie et al introduce learnable attention maps to update the mask dynamically.…”

Section: B Image Inpainting By Deep Generative Modelsmentioning

confidence: 99%

Semantic-SCA: Semantic Structure Image Inpainting With the Spatial-Channel Attention

2021

View full text Add to dashboard Cite

Deep learning has brought unprecedented progress to image inpainting. However, the existing methods often generate images with blurry textures and distorted structures because they may either fail to maintain semantic consistency or restore fine-grained textures. In this paper, we propose a two-stage adversarial model to further improve the accuracy of the structure and details of image inpainting. Our model splits the inpainting task into two parts: semantic structure reconstructor and texture generator. In the first stage, we first utilize the semantic structure map based on the unsupervised segmentation to train the semantic structure reconstructor, which completes the missing structures of the inputs and maintains consistency between the missing part and the overall image. In the second stage, we introduce the spatial-channel attention (SCA) module to obtain the fine-grained textures. The SCA module strengthens the capability to obtain information from the long-distance pixel and different channels of the model. Furthermore, we propose a spatial-channel loss to stabilize the network training process and improve visual effects. Finally, we evaluate our model over the publicly available datasets CelebA, Places2, and Paris StreetView. When the inpainting tasks involved in large-area defects or heavy structure, the experimental results show that our method has a higher inpainting quality than the existing state-of-the-art approaches. INDEX TERMS Artificial neural networks, deep learning, generative model, image generation, image inpainting.

show abstract

“…In [37], a learnable bidirectional attention maps (LBAM) for image inpainting is proposed. The method used FCN to conduct image inpainting.…”

Section: Related Workmentioning

confidence: 99%

Region-Based Removal of Thermal Reflection Using Pruned Fully Convolutional Network

et al. 2020

View full text Add to dashboard Cite

In general, an image obtained from a thermal camera often has a mirror reflection or shadow reflected off the ground around an object, which is referred to as thermal reflection. Sometimes the thermal reflections are connected to their objects in images, which makes it difficult to detect or recognize the object only. Thermal reflections sometimes occur on the wall near an object and are detected as another object when they are not connected to the object. Furthermore, the size of thermal reflection and pixel value significantly vary with the medium of the reflected range and the surrounding temperature. In these cases, the patterns and pixel values of thermal reflection and the object become similar and difficult to distinguish. However, there are insufficient studies on removing the thermal reflection of various kinds of objects in diverse environments. Therefore, in this paper, we propose a pruned fully convolutional network (PFCN)-based method for removing the thermal reflection of an object using the surrounding information when image transformation is performed only within the region of an object. When experiments were conducted using self-collected databases (Dongguk thermal image database (DTh-DB) and Dongguk items & vehicles database (DI&V-DB)) and open databases, the method proposed herein exhibited more outstanding performance in removing thermal reflection when compared with the state-of-the-art methods. INDEX TERMS Thermal image, image transform, thermal reflection removal, pruned fully convolutional network.

show abstract

Image Inpainting With Learnable Bidirectional Attention Maps

Cited by 235 publications

References 31 publications

Image Inpainting Using Two-Stage Loss Function and Global and Local Markovian Discriminators

Image Inpainting Using Two-Stage Loss Function and Global and Local Markovian Discriminators

Semantic-SCA: Semantic Structure Image Inpainting With the Spatial-Channel Attention

Region-Based Removal of Thermal Reflection Using Pruned Fully Convolutional Network

Contact Info

Product

Resources

About