Facial Image Inpainting With Deep Generative Model and Patch Search Using Region Weight

Wei, Jinsheng; Lu, Guanming; Liu, Huaming; Yan, Jun

doi:10.1109/access.2019.2919169

Cited by 11 publications

(4 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For quantitative comparisons, facial inpainting works report results using L1 error, PSNR, SSIM [36], and inception score [31] in general. However, many works [7,18,37] mention that there are no good evaluation metrics due to many possible solutions different from the original image content. These metrics generate scores based on the similarity of the produced results to the original image.…”

Section: Model Comparisonsmentioning

confidence: 99%

Iterative Facial Image Inpainting Based on an Encoder-Generator Architecture

Dogan¹,

Keleş²

2021

Preprint

View full text Add to dashboard Cite

Facial image inpainting is a challenging problem as it requires generating new pixels that include semantic information for masked key components in a face, e.g., eyes and nose. Recently, remarkable methods have been proposed in this field. Most of these approaches use encoder-decoder architectures and have different limitations such as allowing unique results for a given image and a particular mask. Alternatively, some approaches generate promising results using different masks with generator networks. However, these approaches are optimizationbased and usually require quite a number of iterations. In this paper, we propose an efficient solution to the facial image painting problem using the Cyclic Reverse Generator (CRG) architecture, which provides an encodergenerator model. We use the encoder to embed a given image to the generator space and incrementally inpaint the masked regions until a plausible image is generated; a discriminator network is utilized to assess the generated images during the iterations. We empirically observed that only a few iterations are sufficient to generate realistic images with the proposed model. After the generation process, for the post processing, we utilize a Unet model that we trained specifically for this task to remedy the artifacts close to the mask boundaries. Our method allows applying sketch-based inpaintings, using variety of mask types, and producing multiple and diverse results. We qualitatively compared our method with the state-of-the-art models and observed that our method can compete with the other models in all mask types; it is particularly better in images where larger masks are utilized.

show abstract

Section: Model Comparisonsmentioning

confidence: 99%

Iterative Facial Image Inpainting Based on an Encoder-Generator Architecture

Dogan¹,

Keleş²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…When implementing DRS, we follow the setting in [14] and set γ dynamically for each batch of fake samples drawn from the GAN to the 95-th percentile of the F (x) in Eq. (7) for each x in this batch. We also keep training the discriminator on the validation set for another 20 epochs to further improve DRS's performance.…”

Section: A Mixture Of 25 2-d Gaussiansmentioning

confidence: 99%

“…G ENERATIVE ADVERSARIAL NETWORKS (GANs) first introduced by [1] are well-known and powerful generative models for image synthesis and have been applied to various types of image-related tasks [2,3,4,5,6,7]. The vanilla GANs proposed by [1] consist of two neural networks: a generator and a discriminator.…”

Section: Introductionmentioning

confidence: 99%

Subsampling Generative Adversarial Networks: Density Ratio Estimation in Feature Space With Softplus Loss

Ding

Wang

Welch

2020

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

Filtering out unrealistic images from trained generative adversarial networks (GANs) has attracted considerable attention recently. Two density ratio based subsampling methods-Discriminator Rejection Sampling (DRS) and Metropolis-Hastings GAN (MH-GAN)-were recently proposed, and their effectiveness in improving GANs was demonstrated on multiple datasets. However, DRS and MH-GAN are based on discriminator based density ratio estimation (DRE) methods, so they may not work well if the discriminator in the trained GAN is far from optimal. Moreover, they do not apply to some GANs (e.g., MMD-GAN). In this paper, we propose a novel Softplus (SP) loss for DRE. Based on it, we develop a sample-based DRE method in a feature space learned by a specially designed and pre-trained ResNet-34 (DRE-F-SP). We derive the rate of convergence of a density ratio model trained under the SP loss. Then, we propose three different density ratio based subsampling methods (DRE-F-SP+RS, DRE-F-SP+MH, and DRE-F-SP+SIR) for GANs based on DRE-F-SP. Our subsampling methods do not rely on the optimality of the discriminator and are suitable for all types of GANs. We empirically show our subsampling approach can substantially outperform DRS and MH-GAN on a synthetic dataset and the CIFAR-10 dataset, using multiple GANs.

show abstract

“…The feature distribution dataset learned by a neural network is more suitable for facial image restoration with a large missing area and random damage. Not only are the texture details accurate but also are the contours harmonized, and the facial image conforms to the contextual semantics (Wei et al, 2019 ). After ongoing in-depth research by relevant scholars, deep learning-based image repair methods have produced a number of results.…”

Section: Introductionmentioning

confidence: 99%

Facial image inpainting for big data using an effective attention mechanism and a convolutional neural network

Zhao

et al. 2023

Front. Neurorobot.

View full text Add to dashboard Cite

Big data facial image is an important identity information for people. However, facial image inpainting using existing deep learning methods has some problems such as insufficient feature mining and incomplete semantic expression, leading to output image artifacts or fuzzy textures. Therefore, it is of practical significance to study how to effectively restore an incomplete facial image. In this study, we proposed a facial image inpainting method using a multistage generative adversarial network (GAN) and the global attention mechanism (GAM). For the overall network structure, we used the GAN as the main body, then we established skip connections to optimize the network structure, and used the encoder–decoder structure to better capture the semantic information of the missing part of a facial image. A local refinement network has been proposed to enhance the local restoration effect and to weaken the influence of unsatisfactory results. Moreover, GAM is added to the network to magnify the interactive features of the global dimension while reducing information dispersion, which is more suitable for restoring human facial information. Comparative experiments on CelebA and CelebA-HQ big datasets show that the proposed method generates realistic inpainting results in both regular and irregular masks and achieves peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), as well as other evaluation indicators that illustrate the performance and efficiency of the proposed model.

show abstract

Facial Image Inpainting With Deep Generative Model and Patch Search Using Region Weight

Cited by 11 publications

References 31 publications

Iterative Facial Image Inpainting Based on an Encoder-Generator Architecture

Iterative Facial Image Inpainting Based on an Encoder-Generator Architecture

Subsampling Generative Adversarial Networks: Density Ratio Estimation in Feature Space With Softplus Loss

Facial image inpainting for big data using an effective attention mechanism and a convolutional neural network

Contact Info

Product

Resources

About