Symmetric Skip Connection Wasserstein GAN for High-Resolution Facial Image Inpainting

et al. 2021

Preprint

Self Cite

Facial image inpainting, with high-fidelity preservation for image realism, is a very challenging task. This is due to the subtle texture in key facial features (component) that are not easily transferable. Many image inpainting techniques have been proposed with outstanding capabilities and high quantitative performances recorded. However, with facial inpainting, the features are more conspicuous and the visual quality of the blended inpainted regions are more important qualitatively. Based on these facts, we design a foreground-guided facial inpainting framework that can extract and generate facial features using convolutional neural network layers. It introduces the use of foreground segmentation masks to preserve the fidelity. Specifically, we propose a new loss function with semantic capability reasoning of facial expressions, natural and unnatural features (make-up). We conduct our experiments using the CelebA-HQ dataset, segmentation masks from CelebAMask-HQ (for foreground guidance) and Quick Draw Mask (for missing regions). Our proposed method achieved comparable quantitative results when compare to the state of the art but qualitatively, it demonstrated high-fidelity preservation of facial components.

Section: Methodsmentioning

confidence: 99%

Section: Foreground Facial Inpainting Frameworkmentioning

confidence: 99%

Foreground-guided Facial Inpainting with Fidelity Preservation

Jam¹,

Kendrick²,

et al. 2021

Preprint

Self Cite

“…By adding the activation layer of a previous layer, to a deeper layer within the network, a residual connection is achieved. In previous works [9,57,17], feature extraction and propagation often fail with large portions of the background due to low level capture and poor transition to the decoder. We consider maxpooling, an operation that highlights the most present feature of an image patch and calculates its maximum value.…”

Section: Recursive Residual Transition Layermentioning

confidence: 99%

“…The learning-based methods, popular known as deep generative neural networks, have become the state of the art, based on their ability to learn distribution with regards to context. These approaches [9,10,11,12,13,14,15,16,17,18,19] use convolutional neural network (CNN) within an encoder-decoder within a GAN-based network to generate realistic images. These algorithms with a wide range of parameters and layers learn to manage feature extraction, propagation, and regularisation.…”

Section: Introductionmentioning

confidence: 99%

V-LinkNet: Learning Contextual Inpainting Across Latent Space of Generative Adversarial Network

Jam¹,

Kendrick²,

et al. 2022

Preprint

Self Cite

Deep learning methods outperform traditional methods in image inpainting. In order to generate contextual textures, researchers are still working to improve on existing methods and propose models that can extract, propagate, and reconstruct features similar to ground-truth regions. Furthermore, the lack of a high-quality feature transfer mechanism in deeper layers contributes to persistent aberrations on generated inpainted regions. To address these limitations, we propose the V-LinkNet cross-space learning strategy network. To improve learning on contextualised features, we design a loss model that employs both encoders. In addition, we propose a recursive residual transition layer (RSTL).The RSTL extracts high-level semantic information and propagates it down layers. Finally, we compare inpainting performance on the same face with different masks and on different faces with the same masks. To improve image inpainting reproducibility, we propose a standard protocol to overcome biases with various masks and images. We investigate the V-LinkNet components using experimental methods. Our result surpasses the state of the art when evaluated on the CelebA-HQ with the standard protocol. In addition, our model can generalise well when evaluated on Paris Street View, and Places2 datasets with the standard protocol.

“…In contrast, the second group of approaches (deep learning methods) [27,13,38,37,23,24,36,39,16] uses generative neural networks to hallucinate missing content of an image based on encoding the semantic context of the image into feature space for realistic output by a decoder. This is done through convolutions which is an operation that extracts feature maps by evaluating the dot product between a kernel and each location of the input image.…”

Section: Introductionmentioning

confidence: 99%

R-MNet: A Perceptual Adversarial Network for Image Inpainting

Jam¹,

Kendrick²,

et al. 2020

Preprint

Self Cite

Facial image inpainting is a problem that is widely studied, and in recent years the introduction of Generative Adversarial Networks, has led to improvements in the field. Unfortunately some issues persists, in particular when blending the missing pixels with the visible ones. We address the problem by proposing a Wasserstein GAN combined with a new reverse mask operator, namely Reverse Masking Network (R-MNet), a perceptual adversarial network for image inpainting. The reverse mask operator transfers the reverse masked image to the end of the encoder-decoder network leaving only valid pixels to be inpainted. Additionally, we propose a new loss function computed in feature space to target only valid pixels combined with adversarial training. These then capture data distributions and generate images similar to those in the training data with achieved realism (realistic and coherent) on the output images. We evaluate our method on publicly available dataset, and compare with state-of-the-art methods. We show that our method is able to generalize to highresolution inpainting task, and further show more realistic outputs that are plausible to the human visual system when compared with the state-of-the-art methods.