Efficient texture-aware multi-GAN for image inpainting

Hedjazi, Mohamed Abbas; Genç, Yakup

doi:10.1016/j.knosys.2021.106789

Cited by 36 publications

(8 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Jam et al 17 proposed using the Wasserstein-perceptual loss function to preserve the image color and maintain the realism of the restored image. Pertinently, Zhang et al 18 independently proposed the WGAN-GP, which was introduced into the global D and local D. Building upon previous work; Hedjazi and Genc 19 proposed optimizing the parameters of four progressively efficient generators and Ds in an end-to-end training approach. Xu et al 20 proposed generating adversarial strategies using reconstructive sampling and multiple granularities.…”

Section: Gan-based Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Hybrid attention generative adversarial network: texture inpainting algorithm for iris defects with excellent repair performance and generalization

Chen

Zeng

et al. 2023

J. Electron. Imag.

View full text Add to dashboard Cite

.Iris defect texture inpainting is a challenging problem that is not only limited by a lack of research but also by requiring a higher degree of texture refinement than other types of images. In the field of image inpainting, most recent research has focused on designing improved encoder-decoder models. Solving the image blurring problem caused by autoencoder models has become a key factor in judging their merits. Generative adversarial networks were designed based on a hybrid attention generative adversarial network (Hybrid A-GAN) mechanism to repair missing iris textures. The generator is based on the encoder-decoder structure and introduces two attention mechanisms to enable obtaining the correlation between channels in the feature map and pixel importance in space, which enhances the network’s ability to utilize feature information. Moreover, the improved jump connection effectively fuses the high-level features with the low-level features after weighting, which prevents the information loss caused by the downsampling process and enhances the image generation capability. In addition, the joint Wasserstein generative adversarial network-gradient penalty and L1 loss jointly guide network training, which further enhances network generation performance and ensures global consistency of the generated images. Extensive repair experiments and recognition experiments conducted on three publicly available datasets demonstrated that hybrid A-GAN has excellent repair capability and generalization performance.

show abstract

Section: Gan-based Methodsmentioning

confidence: 99%

“…Pertinently, Zhang et al 18 . independently proposed the WGAN-GP, which was introduced into the global D and local D. Building upon previous work; Hedjazi and Genc 19 proposed optimizing the parameters of four progressively efficient generators and Ds in an end-to-end training approach. Xu et al 20 .…”

Section: Related Workmentioning

confidence: 99%

Hybrid attention generative adversarial network: texture inpainting algorithm for iris defects with excellent repair performance and generalization

Chen

Zeng

et al. 2023

J. Electron. Imag.

View full text Add to dashboard Cite

show abstract

“…For text and image modalities, the diffusion model [ 26 – 28 ] can learn the denoising process while allowing conditional guidance to flexibly adapt to the semantic reconstruction. The audio and haptic modalities cannot be processed directly so that the authors propose GANs [ 20 , 29 , 30 ] to reconstruct their spectrum into new signals, quickly [ 19 , 31 , 32 ]. Hence, generative AI improves efficiency of semantic codec [ 33 – 35 ], accuracy of semantic transmission, and creativity of semantic reconstruction.…”

Section: Introductionmentioning

confidence: 99%

Cross-Modal Graph Semantic Communication Assisted by Generative AI in the Metaverse for 6G

Chen,

Liu,

Wang

et al. 2024

Research

View full text Add to dashboard Cite

Recently, the development of the Metaverse has become a frontier spotlight, which is an important demonstration of the integration innovation of advanced technologies in the Internet. Moreover, artificial intelligence (AI) and 6G communications will be widely used in our daily lives. However, the effective interactions with the representations of multimodal data among users via 6G communications is the main challenge in the Metaverse. In this work, we introduce an intelligent cross-modal graph semantic communication approach based on generative AI and 3-dimensional (3D) point clouds to improve the diversity of multimodal representations in the Metaverse. Using a graph neural network, multimodal data can be recorded by key semantic features related to the real scenarios. Then, we compress the semantic features using a graph transformer encoder at the transmitter, which can extract the semantic representations through the cross-modal attention mechanisms. Next, we leverage a graph semantic validation mechanism to guarantee the exactness of the overall data at the receiver. Furthermore, we adopt generative AI to regenerate multimodal data in virtual scenarios. Simultaneously, a novel 3D generative reconstruction network is constructed from the 3D point clouds, which can transfer the data from images to 3D models, and we infer the multimodal data into the 3D models to increase realism in virtual scenarios. Finally, the experiment results demonstrate that cross-modal graph semantic communication, assisted by generative AI, has substantial potential for enhancing user interactions in the 6G communications and Metaverse.

show abstract

“…Among them, the method based on Generative Adversarial Nets [3] (GANs) had become the mainstream in the field of image repair [4]. e method based on GANs transforms the image repair problem into a condition-based generation confrontation [5,6]. Such methods usually take the damaged image and the mask of the calibrated damaged area as the conditional input, use the autoencoder network as the generator to reconstruct the content of the damaged area, and combine the discriminator network to counteract training, and finally get a complete image output [7].…”

Section: Introductionmentioning

confidence: 99%

Research into an Image Inpainting Algorithm via Multilevel Attention Progression Mechanism

Chen

2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Existing image inpainting schemes generally have the problems of structural disorder and blurred texture details. This is mainly because, in the reconstruction process of the damaged area of the image, it is difficult for the inpainting network to make full use of the information in the nondamaged area to accurately infer the content of the damaged area. Therefore, the paper has proposed an image inpainting network driven by multilevel attention progression mechanism. The proposed network has compressed the high-level features extracted from the full-resolution image into multiscale compact features and then drives the compact features to perform multilevel order according to the scale size. Attention feature progression is to achieve the goal of the full progression of high-level features including structure and details in the network. To further realize fine-grained image inpainting and reconstruction, the paper has also proposed a composite granular discriminator to achieve image inpainting process performing global semantic constraints and nonspecific local dense constraints. The related experimental results in the paper can show that the proposed method can achieve higher quality repair results than state-of-the-art ones.

show abstract

Efficient texture-aware multi-GAN for image inpainting

Cited by 36 publications

References 30 publications

Hybrid attention generative adversarial network: texture inpainting algorithm for iris defects with excellent repair performance and generalization

Hybrid attention generative adversarial network: texture inpainting algorithm for iris defects with excellent repair performance and generalization

Cross-Modal Graph Semantic Communication Assisted by Generative AI in the Metaverse for 6G

Research into an Image Inpainting Algorithm via Multilevel Attention Progression Mechanism

Contact Info

Product

Resources

About