Dual Projection Generative Adversarial Networks for Conditional Image Generation

Han, Ligong; Min, Martin Renqiang; Stathopoulos, Anastasis; Tian, Yu; Gao, Ruohan; Kadav, Asim; Metaxas, Dimitris N.

doi:10.1109/iccv48922.2021.01417

Cited by 19 publications

(13 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Conditional GANs (cGANs) [48] are variants of GANs that synthesize realistic and diverse images using conditional information, such as class labels. Depending on how the framework incorporates the data and class labels, most cGANs can be categorized into classifier-based cGANs [19], [36], [37], [54] and projection-based cGANs [4], [24], [49], [50]. Classifierbased cGANs utilize conditional information (class labels) by training an additional classifier as well as a standard GAN discriminator.…”

Section: B Conditional Generative Adversarial Networkmentioning

confidence: 99%

Continuous Facial Motion Deblurring

2022

View full text Add to dashboard Cite

We introduce a novel framework for continuous facial motion deblurring that restores the continuous sharp moment latent in a single motion-blurred face image via a moment control factor. Although a motion-blurred image is the accumulated signal of continuous sharp moments during the exposure time, most existing single image deblurring approaches aim to restore a fixed number of frames using multiple networks and training stages. To address this problem, we propose a continuous facial motion deblurring network based on GAN (CFMD-GAN), which is a novel framework for restoring the continuous moment latent in a single motion-blurred face image with a single network and a single training stage. To stabilize the network training, we train the generator to restore continuous moments in the order determined by our facial motion-based reordering process (FMR) utilizing domain-specific knowledge of the face. Moreover, we propose an auxiliary regressor that helps our generator produce more accurate images by estimating continuous sharp moments. Furthermore, we introduce a control-adaptive (ContAda) block that performs spatially deformable convolution and channel-wise attention as a function of the control factor. Extensive experiments on the 300VW datasets demonstrate that the proposed framework generates a various number of continuous output frames by varying the moment control factor. Compared with the recent single-tosingle image deblurring networks trained with the same 300VW training set, the proposed method show the superior performance in restoring the central sharp frame in terms of perceptual metrics, including LPIPS, FID and Arcface identity distance. The proposed method outperforms the existing single-to-video deblurring method for both qualitative and quantitative comparisons. In our experiments on the 300VW test set, the proposed framework reached 33.14 dB and 0.93 for recovery of 7 sharp frames in PSNR and SSIM, respectively.

show abstract

Section: B Conditional Generative Adversarial Networkmentioning

confidence: 99%

Continuous Facial Motion Deblurring

2022

View full text Add to dashboard Cite

show abstract

“…For example, from a horse to a zebra, from a low-resolution image to a high-resolution image, from a photograph to an art painting, and vice versa [1,2]. UI2I has received a lot of attention due to its excellent performance in areas such as image style transfer [3][4][5][6][7], colourisation [8], super-resolution [9,10], dehazing [11], denoising [12], image Synthesis [13], text-to-image Synthesis [14], image Generation [15,16], and underwater image restoration [17].…”

Section: Introductionmentioning

confidence: 99%

CRPGAN: Learning image-to-image translation of two unpaired images by cross-attention mechanism and parallelization strategy

Feng

Geng

et al. 2023

PLoS ONE

View full text Add to dashboard Cite

Unsupervised image-to-image translation (UI2I) tasks aim to find a mapping between the source and the target domains from unpaired training data. Previous methods can not effectively capture the differences between the source and the target domain on different scales and often leads to poor quality of the generated images, noise, distortion, and other conditions that do not match human vision perception, and has high time complexity. To address this problem, we propose a multi-scale training structure and a progressive growth generator method to solve UI2I task. Our method refines the generated images from global structures to local details by adding new convolution blocks continuously and shares the network parameters in different scales and also in the same scale of network. Finally, we propose a new Cross-CBAM mechanism (CRCBAM), which uses a multi-layer spatial attention and channel attention cross structure to generate more refined style images. Experiments on our collected Opera Face, and other open datasets Summer↔Winter, Horse↔Zebra, Photo↔Van Gogh, show that the proposed algorithm is superior to other state-of-art algorithms.

show abstract

“…For the joint distribution modeling of data and additional information such as labels, conditional GANs (cGANs) are widely used [21], [28], [29], [30], [31], [32], [33], [34], [35]. By incorporating data and additional labels, the discriminators identify real images in a principled way, thereby resulting in generators that produce realistic images [36], [37].…”

Section: Introductionmentioning

confidence: 99%

“…By incorporating data and additional labels, the discriminators identify real images in a principled way, thereby resulting in generators that produce realistic images [36], [37]. Recently, projection GANs [28], [33], [34], [35] have successfully decomposed joint distributions into image distribution (marginal) and label matching distribution (conditional). Specifically, as shown in Fig.…”

Section: Introductionmentioning

confidence: 99%

Semantic-Aware Face Deblurring With Pixel-Wise Projection Discriminator

2023

View full text Add to dashboard Cite

Most recent face deblurring methods have leveraged the distribution modeling ability of generative adversarial networks (GANs) to impose a constraint that the deblurred image should follow the distribution of sharp ground-truth images. However, generating sharp face images with high fidelity and realistic properties from a blurry face image remains challenging under the GAN framework. To this end, we focus on modeling the joint distribution of sharp face images and segmentation label maps for face image deblurring in a GAN framework. We propose a semantic-aware pixel-wise projection (SAPP) discriminator that models pixel-label matching with semantic label map information and generates a pixel-wise probability map of realness for the input image as well as a per-image probability. Moreover, we introduce a predictionweighted (PW) loss to focus on erroneous pixels in the output of the decoder, using per-pixel real/fake probability map to re-weight the contribution of each pixel in the decoder. Furthermore, we present a coarseto-fine training technique for the generator, which encourages the generator to focus on global consistency in the early training stages and local details in the later stages. Extensive experimental results show that our method outperforms existing methods both quantitatively and qualitatively in terms of perceptual image quality.INDEX TERMS Face image deblurring, semantic-aware pixel-wise projection discriminator, predictionweighted loss.

show abstract

Dual Projection Generative Adversarial Networks for Conditional Image Generation

Cited by 19 publications

References 8 publications

Continuous Facial Motion Deblurring

Continuous Facial Motion Deblurring

CRPGAN: Learning image-to-image translation of two unpaired images by cross-attention mechanism and parallelization strategy

Semantic-Aware Face Deblurring With Pixel-Wise Projection Discriminator

Contact Info

Product

Resources

About