R-MNet: A Perceptual Adversarial Network for Image Inpainting

Jam, Jireh; Kendrick, Connah; Drouard, Vincent; Walker, Kevin; Hsu, Gee-Sern; Yap, Moi Hoon

doi:10.1109/wacv48630.2021.00276

Cited by 24 publications

(16 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Also, 3GAN (Kottler et al, 2022) proposed to reconstruct the hidden parts of the building Walls and remove foreground objects like trees and signs. E2F-GAN (Hassanpour et al, 2022), R-MNet (Jam et al, 2021), and 3DMM-conditioned GAN (X. Yuan & Park, 2019) were proposed to fill and recover the damaged parts of the faces images.…”

Section: Image Inpaintingmentioning

confidence: 99%

See 1 more Smart Citation

A comprehensive review of generative adversarial networks: Fundamentals, applications, and challenges

Mohammed

2023

WIREs Computational Stats

View full text Add to dashboard Cite

In machine learning, a generative model is responsible for generating new samples of data in terms of a probabilistic model. Generative adversarial network (GAN) has been widely used to generate realistic samples in different domains and outperforms its peers in the generative models family. However, producing a robust GAN model is not a trivial task because many challenges face the GAN during the training process and impact its performance, affecting the quality and diversity of the generated samples. In this article, we conduct a comprehensive review of GANs to present the fundamentals of GAN, including its components, types, and objective functions. Also, we present an overview of the evaluation matrices used to evaluate GAN models. Moreover, we list the applications of GANs and research work in various domains. Finally, we present the challenges that face GANs and highlight two significant issues, representing mode collapse and training instability, in addition to those research efforts that tackle these challenges.This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Deep Learning Statistical Learning and Exploratory Methods of the Data Sciences > Neural Networks

show abstract

Section: Image Inpaintingmentioning

confidence: 99%

“…F I G U R E 5 Illustration of recovering the corrupted parts in the images(Jam et al, 2021;Kottler et al, 2022).F I G U R E 6 Illustration of image-to-image translation (J.-Y Zhu et al, 2017)…”

mentioning

confidence: 99%

A comprehensive review of generative adversarial networks: Fundamentals, applications, and challenges

Mohammed

2023

WIREs Computational Stats

View full text Add to dashboard Cite

show abstract

“…The generative adversarial network (GAN) was introduced by Goodfellow et al [5] to produce realistic images under certain conditions. GANs have attracted substantial attention and has been studied in many tasks [17], such as image synthesis [5,13,14,21], text-to-image translation [22,36], and image inpainting [11,15,16]. In this work, we focus on talking head video generation with GAN guided by 3D facial depth maps learned without any ground-truth depths.…”

Section: Related Workmentioning

confidence: 99%

Depth-Aware Generative Adversarial Network for Talking Head Video Generation

Hong¹,

Zhang²,

Shen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Talking head video generation aims to produce a synthetic human face video that contains the identity and pose information respectively from a given source image and a driving video. Existing works for this task heavily rely on 2D representations (e.g. appearance and motion) learned from the input images. However, dense 3D facial geometry (e.g. pixel-wise depth) is extremely important for this task as it is particularly beneficial for us to essentially generate accurate 3D face structures and distinguish noisy information from the possibly cluttered background. Nevertheless, dense 3D geometry annotations are prohibitively costly for videos and are typically not available for this video generation task. In this paper, we first introduce a self-supervised geometry learning method to automatically recover the dense 3D geometry (i.e. depth) from the face videos without the requirement of any expensive 3D annotation data. Based on the learned dense depth maps, we further propose to leverage them to estimate sparse facial keypoints that capture the critical movement of the human head. In a more dense way, the depth is also utilized to learn 3D-aware cross-modal (i.e. appearance and depth) attention to guide the generation of motion fields for warping source image representations. All these contributions compose a novel depth-aware generative adversarial network (DaGAN) for talking head generation. Extensive experiments conducted demonstrate that our proposed method can generate highly realistic faces, and achieve significant results on the unseen human faces. 1

show abstract

“…I MAGE inpainting aims to restore missing regions of corrupted images with realistic content, and has a wide range of applications in photo editing, de-captioning and other scenarios where people might want to remove unwanted objects from their photos [17], [34], [37], [48]. Recent image inpainting models usually rely on complicated neural networks and well-designed loss functions to produce satisfactory results [29], [42].…”

Section: Introductionmentioning

confidence: 99%

Improve Deep Image Inpainting by Emphasizing the Complexity of Missing Regions

Wang¹,

Li²,

Xu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep image inpainting research mainly focuses on constructing various neural network architectures or imposing novel optimization objectives. However, on the one hand, building a state-of-the-art deep inpainting model is an extremely complex task, and on the other hand, the resulting performance gains are sometimes very limited. We believe that besides the frameworks of inpainting models, lightweight traditional image processing techniques, which are often overlooked, can actually be helpful to these deep models. In this paper, we enhance the deep image inpainting models with the help of classical image complexity metrics. A knowledge-assisted index composed of missingness complexity and forward loss is presented to guide the batch selection in the training procedure. This index helps find samples that are more conducive to optimization in each iteration and ultimately boost the overall inpainting performance. The proposed approach is simple and can be plugged into many deep inpainting models by changing only a few lines of code. We experimentally demonstrate the improvements for several recently developed image inpainting models on various datasets.

show abstract

R-MNet: A Perceptual Adversarial Network for Image Inpainting

Cited by 24 publications

References 29 publications

A comprehensive review of generative adversarial networks: Fundamentals, applications, and challenges

A comprehensive review of generative adversarial networks: Fundamentals, applications, and challenges

Depth-Aware Generative Adversarial Network for Talking Head Video Generation

Improve Deep Image Inpainting by Emphasizing the Complexity of Missing Regions

Contact Info

Product

Resources

About