PEPSI : Fast Image Inpainting With Parallel Decoding Network

Sagong, Min-Cheol; Shin, Yong-Goo; Kim, Seung-Wook; Park, Seung; Ko, Sung-Jea

doi:10.1109/cvpr.2019.01162

Cited by 129 publications

(67 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, this method needs a lot of computational resources. For this reason, Sagong et al [35] proposed a structure (Pepsi) composed of a single shared coding network and a parallel decoding network with rough and patching paths, which can reduce the number of convolution operations. Recently, some works [20], [23], [29] have proposed the use of spatial attention [24], [25] to obtain high-frequency details.…”

Section: B Image Inpainting By Deep Generative Modelsmentioning

confidence: 99%

PiiGAN: Generative Adversarial Networks for Pluralistic Image Inpainting

2020

View full text Add to dashboard Cite

The latest methods based on deep learning have achieved amazing results regarding the complex work of inpainting large missing areas in an image. But this type of method generally attempts to generate one single ''optimal'' result, ignoring many other plausible results. Considering the uncertainty of the inpainting task, one sole result can hardly be regarded as a desired regeneration of the missing area. In view of this weakness, which is related to the design of the previous algorithms, we propose a novel deep generative model equipped with a brand new style extractor which can extract the style feature (latent vector) from the ground truth. Once obtained, the extracted style feature and the ground truth are both input into the generator. We also craft a consistency loss that guides the generated image to approximate the ground truth. After iterations, our generator is able to learn the mapping of styles corresponding to multiple sets of vectors. The proposed model can generate a large number of results consistent with the context semantics of the image. Moreover, we evaluated the effectiveness of our model on three datasets, i.e., CelebA, PlantVillage, and MauFlex. Compared to state-of-the-art inpainting methods, this model is able to offer desirable inpainting results with both better quality and higher diversity. The code and model will be made available on https://github.com/vivitsai/PiiGAN. INDEX TERMS Deep learning, generative adversarial networks, image inpainting, diversity inpainting.

show abstract

Section: B Image Inpainting By Deep Generative Modelsmentioning

confidence: 99%

PiiGAN: Generative Adversarial Networks for Pluralistic Image Inpainting

2020

View full text Add to dashboard Cite

show abstract

“…However, once the wrong information is captured in the first stage, it will cause the propagation of the error. On this basis, Sagong et al [ 16 ] proposed a parallel extended-decoder path with a modified contextual attention module to reduce the number of convolution operations and create a higher-quality inpainting result simultaneously. For capturing long-range spatial dependencies, the self-attention mechanism [ 17 ] based on the non-local network [ 18 ] was wildly adopted.…”

Section: Related Workmentioning

confidence: 99%

Non-Local and Multi-Scale Mechanisms for Image Inpainting

He¹,

Yin²

2021

Sensors

View full text Add to dashboard Cite

Recently, deep learning-based techniques have shown great power in image inpainting especially dealing with squared holes. However, they fail to generate plausible results inside the missing regions for irregular and large holes as there is a lack of understanding between missing regions and existing counterparts. To overcome this limitation, we combine two non-local mechanisms including a contextual attention module (CAM) and an implicit diversified Markov random fields (ID-MRF) loss with a multi-scale architecture which uses several dense fusion blocks (DFB) based on the dense combination of dilated convolution to guide the generative network to restore discontinuous and continuous large masked areas. To prevent color discrepancies and grid-like artifacts, we apply the ID-MRF loss to improve the visual appearance by comparing similarities of long-distance feature patches. To further capture the long-term relationship of different regions in large missing regions, we introduce the CAM. Although CAM has the ability to create plausible results via reconstructing refined features, it depends on initial predicted results. Hence, we employ the DFB to obtain larger and more effective receptive fields, which benefits to predict more precise and fine-grained information for CAM. Extensive experiments on two widely-used datasets demonstrate that our proposed framework significantly outperforms the state-of-the-art approaches both in quantity and quality.

show abstract

“…Finally, the generator can be used to automatically generate images that are very similar to the real images. Sagong et al [32] proposed a fast PEPSI (Parallel Extended-decoder Path for Semantic Inpainting) model. It can reduce the number of convolution operations by adopting a structure consisting of a single shared encoding network and a parallel decoding network with coarse and inpainting paths.…”

Section: Related Workmentioning

confidence: 99%

An image inpainting method for object removal based on difference degree constraint

Zhang

Chang

2020

Multimed Tools Appl

View full text Add to dashboard Cite

In the inpainting method for object removal, SSD (Sum of Squared Differences) is commonly used to measure the degree of similarity between the exemplar patch and the target patch, which has a very important impact on the restoration results. Although the matching rule is relatively simple, it is likely to lead to the occurrence of mismatch error. Even worse, the error may be accumulated along with the process continues. Finally some unexpected objects may be introduced into the target region, making the result unable to meet the requirements of visual consistency. In view of these problems, we propose an inpainting method for object removal based on difference degree constraint. Firstly, we define the MSD (Mean of Squared Differences) and use it to measure the degree of differences between corresponding pixels at known positions in the target patch and the exemplar patch. Secondly, we define the SMD (Square of Mean Differences) and use it to measure the degree of differences between the pixels at known positions in the target patch and the pixels at unknown positions in the exemplar patch. Thirdly, based on MSD and SMD, we define a new matching rule and use it to find the most similar exemplar patch in the source region. Finally, we use the exemplar patch to restore the target patch. Experimental results show that the proposed method can effectively prevent the occurrence of mismatch error and improve the restoration effect.

show abstract

PEPSI : Fast Image Inpainting With Parallel Decoding Network

Cited by 129 publications

References 17 publications

PiiGAN: Generative Adversarial Networks for Pluralistic Image Inpainting

PiiGAN: Generative Adversarial Networks for Pluralistic Image Inpainting

Non-Local and Multi-Scale Mechanisms for Image Inpainting

An image inpainting method for object removal based on difference degree constraint

Contact Info

Product

Resources

About