Image Inpainting for Irregular Holes Using Partial Convolutions

Liu, Guilin; Reda, Fitsum A.; Shih, Kevin J.; Wang, Ting-Chun; Tao, Andrew; Catanzaro, Bryan

doi:10.48550/arxiv.1804.07723

Cited by 49 publications

(198 citation statements)

References 24 publications

Supporting

Mentioning

168

Contrasting

Unclassified

Order By: Relevance

“…Other methods [35,25,18] observed that failures in feature extraction and propagation could be due to the irregular holes. To address the limitation, Liu et al [11] proposed an independent mask updating with partial convolutions to specifically target missing regions. Yu et al [25] proposed to use gated convolutions to gear the model towards learning soft mask of the irregular hole regions.…”

Section: Learning-based Methodsmentioning

confidence: 99%

“…Previous works [9,10,11,39] in computer vision have shown that inpainting is a learning problem that can be solved by encoding high-level features. The reconstructed output is geared towards having a close similarity to the input.…”

Section: Problem Formulationmentioning

confidence: 99%

“…• CelebA-HQ [62]: A dataset curated from CelebA [63] containing 30000 high quality images of 1024 × 1024, 512 × 512, 256 × 256 and 128 × 128 resolutions. We followed the state-of-the-art procedures [11] and split our dataset into 27K training and 3K testing set.…”

Section: Datasetsmentioning

confidence: 99%

“…The learning-based methods, popular known as deep generative neural networks, have become the state of the art, based on their ability to learn distribution with regards to context. These approaches [9,10,11,12,13,14,15,16,17,18,19] use convolutional neural network (CNN) within an encoder-decoder within a GAN-based network to generate realistic images. These algorithms with a wide range of parameters and layers learn to manage feature extraction, propagation, and regularisation.…”

Section: Introductionmentioning

confidence: 99%

“…It is obvious that GAN-based inpainting methods [9,11,25,18,16,19] have made tremendous strides in terms of visual quality, as measured by the popular Structure Similarity Index Measure (SSIM) [26] and other quantitative evaluations [27,28,29,30]. However, the achieved advancement, which is always judged over existing models by the aforementioned measures, is not as significant as projected.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

V-LinkNet: Learning Contextual Inpainting Across Latent Space of Generative Adversarial Network

Jam¹,

Kendrick²,

Drouard³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep learning methods outperform traditional methods in image inpainting. In order to generate contextual textures, researchers are still working to improve on existing methods and propose models that can extract, propagate, and reconstruct features similar to ground-truth regions. Furthermore, the lack of a high-quality feature transfer mechanism in deeper layers contributes to persistent aberrations on generated inpainted regions. To address these limitations, we propose the V-LinkNet cross-space learning strategy network. To improve learning on contextualised features, we design a loss model that employs both encoders. In addition, we propose a recursive residual transition layer (RSTL).The RSTL extracts high-level semantic information and propagates it down layers. Finally, we compare inpainting performance on the same face with different masks and on different faces with the same masks. To improve image inpainting reproducibility, we propose a standard protocol to overcome biases with various masks and images. We investigate the V-LinkNet components using experimental methods. Our result surpasses the state of the art when evaluated on the CelebA-HQ with the standard protocol. In addition, our model can generalise well when evaluated on Paris Street View, and Places2 datasets with the standard protocol.

show abstract

Section: Learning-based Methodsmentioning

confidence: 99%

Section: Problem Formulationmentioning

confidence: 99%

Section: Datasetsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

V-LinkNet: Learning Contextual Inpainting Across Latent Space of Generative Adversarial Network

Jam¹,

Kendrick²,

Drouard³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

The Unreasonable Effectiveness of Texture Transfer for Single Image Super-Resolution

Gondal

Schölkopf

Hirsch

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

While implicit generative models such as GANs have shown impressive results in high quality image reconstruction and manipulation using a combination of various losses, we consider a simpler approach leading to surprisingly strong results. We show that texture loss[1] alone allows the generation of perceptually high quality images. We provide a better understanding of texture constraining mechanism and develop a novel semantically guided texture constraining method for further improvement. Using a recently developed perceptual metric employing "deep features" and termed LPIPS [2], the method obtains state-of-the-art results. Moreover, we show that a texture representation of those deep features better capture the perceptual quality of an image than the original deep features. Using texture information, off-the-shelf deep classification networks (without training) perform as well as the best performing (tuned and calibrated) LPIPS metrics. The code is publicly available.

show abstract

Road Layout Understanding by Generative Adversarial Inpainting

Berlincioni

Becattini

Galteri

et al. 2019

Inpainting and Denoising Challenges

View full text Add to dashboard Cite

Autonomous driving is becoming a reality, yet vehicles still need to rely on complex sensor fusion to understand the scene they act in. The ability to discern static environment and dynamic entities provides a comprehension of the road layout that poses constraints to the reasoning process about moving objects. We pursue this through a GAN-based semantic segmentation inpainting model to remove all dynamic objects from the scene and focus on understanding its static components such as streets, sidewalks and buildings. We evaluate this task on the Cityscapes dataset and on a novel synthetically generated dataset obtained with the CARLA simulator and specifically designed to quantitatively evaluate semantic segmentation inpaintings. We compare our methods with a variety of baselines working both in the RGB and segmentation domains.

show abstract

Image Inpainting for Irregular Holes Using Partial Convolutions

Cited by 49 publications

References 24 publications

V-LinkNet: Learning Contextual Inpainting Across Latent Space of Generative Adversarial Network

V-LinkNet: Learning Contextual Inpainting Across Latent Space of Generative Adversarial Network

The Unreasonable Effectiveness of Texture Transfer for Single Image Super-Resolution

Road Layout Understanding by Generative Adversarial Inpainting

Contact Info

Product

Resources

About