Reference-Guided Texture and Structure Inference for Image Inpainting

Liu, Taorong; Liao, Liang; Wang, Zheng; Satoh, Shin’ichi

doi:10.1109/icip46576.2022.9897592

Cited by 8 publications

(2 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Under the guidance of face example images with rich texture and semantic information, EXE-GAN can effectively generate more realistic face restoration results. And Liu Taorong et al encoded texture and structural features of input images and reference images and performing multi-scale fusion [41], which achieved more realistic detailed restoration of natural scene images.…”

Section: Priori Guided Image Inpaintingmentioning

confidence: 99%

EA-GAN: restoration of text in ancient Chinese books based on an example attention generative adversarial network

Wenjun

Ruiqi

et al. 2023

Herit Sci

View full text Add to dashboard Cite

Ancient Chinese books are of great significance to historical research and cultural inheritance. Unfortunately, many of these books have been damaged and corroded in the process of long-term transmission. The restoration by digital preservation of ancient books is a new method of conservation. Traditional character restoration methods ensure the visual consistency of character images through character features and the pixels around the damaged area. However, reconstructing characters often causes errors, especially when there is large damage in critical locations. Inspired by human’s imitation writing behavior, a two-branch structure character restoration network EA-GAN (Example Attention Generative Adversarial Network) is proposed, which is based on a generative adversarial network and fuses reference examples. By referring to the features of the example character, the damaged character can be restored accurately even when the damaged area is large. EA-GAN first uses two branches to extract the features of the damaged and example characters. Then, the damaged character is restored according to neighborhood information and features of the example character in different scales during the up-sampling stage. To solve problems when the example and damaged character features are not aligned and the convolution receptive field is too small, an Example Attention block is proposed to assist in restoration. Qualitative and quantitative analysis experiments are carried out on a self-built dataset MSACCSD and real scene pictures. Compared with current inpainting networks, EA-GAN can get the correct text structure through the guidance of the additional example in the Example Attention block. The peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) value increased by 9.82% and 1.82% respectively. The learned perceptual image patch similarity (LPIPS) value calculated by Visual Geometry Group (VGG) network and AlexNet decreased by 35.04% and 16.36% respectively. Our method obtained better results than the current inpainting methods. It also has a good restoration effect in the face of untrained characters, which is helpful for the digital preservation of ancient Chinese books.

show abstract

Section: Priori Guided Image Inpaintingmentioning

confidence: 99%

EA-GAN: restoration of text in ancient Chinese books based on an example attention generative adversarial network

Wenjun

Ruiqi

et al. 2023

Herit Sci

View full text Add to dashboard Cite

show abstract

“…Lu Wanglong et al proposed EXE-GAN [40] in 2022, which can effectively generate more realistic face repair results under the guidance of face example images with rich texture and semantic information. And Liu Taorong et al encoded texture and structural features of input images and reference images and performing multi-scale fusion [41], which achieved more realistic detailed restoration of natural scene images.…”

Section: Priori Guided Image Inpaintingmentioning

confidence: 99%

EA-GAN: Ancient books text restoration model based on example attention

Wenjun

Feng

et al. 2022

Preprint

View full text Add to dashboard Cite

Ancient books are of great significance to historical research and cultural inheritance. Unfortunately, these books have been damaged and corroded in the process of long-term transmission. The restoration and digital preservation of ancient books is a new protection method. Traditional character restoration methods ensure the visual consistency of character images through the character features and the pixels around the damaged hole. However, character structure often causes errors, especially when there is a big hole in critical locations. Inspired by human copywriting behavior, a two-branch structure character restoration network EA-GAN is proposed, which is based on generative adversarial network and fuses reference examples. By referring to the feature of the example character, the damaged character can be repaired accurately even when the damaged area is large. EA-GAN first uses two branches to extract the features of the damaged character and example character respectively. Then, the damaged character is repaired according to their neighborhood information and example character features of different scales in the upsampling stage. To solve the problems that the example features and the fixed character features are not aligned and the convolution receptive field is too small, an Example Attention structure block is proposed to assin ist repair. Qualitative and quantitative analysis experiments are carried out on the self-built dataset MSACCSD and real scene pictures. Compared with the latest repair networks, EA-GAN can get the correct text structure through the guidance of the additional example in the Example Attention block. The peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) value were improved by 9.82% and 1.82%, and the learned perceptual image patch similarity (LPIPS) value calculated by VGG and AlexNet decreased by 35.04% and 16.36% respectively, which obtained better results than the current repair methods. It also has a good repair effect in the face of untrained characters ,which is helpful to the digital preservation of ancient books.

show abstract