Pairwise-GAN: Pose-Based View Synthesis Through Pair-Wise Training

Shen, Xuyang; Plested, Jo; Ya, Yao; Gedeon, Tom

doi:10.1007/978-3-030-63820-7_58

Cited by 5 publications

(9 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With the publication of TP-GAN (Two Pathways Generative Adversarial Networks) and CR-GAN (Complete Representations Generative Adversarial Networks), the research on face frontalization shifted into conditional-GANs [10], [22]. Since the high achievement in image translation by Pix2Pix (Pixel to Pixel Generative Adversarial Networks) and CycleGAN (Cycle-Consistent Adversarial Networks) [4], [5], recent publications in face frontalization were based on these two frameworks [3], [6], [11], [28]. The current state-of-the-art model in face frontaliztaion using the Color FRET database is Pairwise-GAN (Pairwise Generative Adversarial Networks), constructed by pair generators and a PatchGAN as the discriminator [3].…”

Section: B Face Frontalizationmentioning

confidence: 99%

“…Since the high achievement in image translation by Pix2Pix (Pixel to Pixel Generative Adversarial Networks) and CycleGAN (Cycle-Consistent Adversarial Networks) [4], [5], recent publications in face frontalization were based on these two frameworks [3], [6], [11], [28]. The current state-of-the-art model in face frontaliztaion using the Color FRET database is Pairwise-GAN (Pairwise Generative Adversarial Networks), constructed by pair generators and a PatchGAN as the discriminator [3]. Compared to other models, the authors proposed to split two domains (left pose and right pose) to synthesize faces.…”

Section: B Face Frontalizationmentioning

confidence: 99%

“…With these properties, the representation bias is significantly reduced. In [3], we published the first version of pre-processing FERET data through MTCNN, which they did not achieve ideal states as MTCNN frequently fails in detecting side-pose faces. We selected Face++ (a commercial face analysis API) to crop and resize images.…”

Section: A Datasetmentioning

confidence: 99%

“…Paying attention to learning algorithms' bias should also be an obligation for computer science researchers from both ethical and technical perspectives. Based on these motivations, we extend our previous work on facial frontalization by exploring the biases from a semantic latent space perspective [3]. Facial frontalization refers to recovering the frontal face from a sidepose image, and belongs to the novel view synthesis domain.…”

Section: Introductionmentioning

confidence: 97%

“…Two base frameworks in facial frontalization from previous research are Pix2Pix (Pixel to Pixel generative adversarial networks) and Cycle-GAN (Cycle-Consistent generative adversarial networks) [4], [5]. The current state-of-the-art models are Pairwise-GAN and FFWM (Flow-based Feature Warping Models), and use different databases [3], [6]. We selected Pix2Pix and Pairwise-GAN for our analysis.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Exploring Biases and Prejudice of Facial Synthesis via Semantic Latent Space

Shen

Plested

Caldwell

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Deep learning (DL) models are widely used to provide a more convenient and smarter life. However, biased algorithms will negatively influence us. For instance, groups targeted by biased algorithms will feel unfairly treated and even fearful of negative consequences of these biases. This work targets biased generative models' behaviors, identifying the cause of the biases and eliminating them. We can (as expected) conclude that biased data causes biased predictions of face frontalization models. Varying the proportions of male and female faces in the training data can have a substantial effect on behavior on the test data: we found that the seemingly obvious choice of 50:50 proportions was not the best for this dataset to reduce biased behavior on female faces, which was 71% unbiased as compared to our top unbiased rate of 84%. Failure in generation and generating incorrect gender faces are two behaviors of these models. In addition, only some layers in face frontalization models are vulnerable to biased datasets. Optimizing the skipconnections of the generator in face frontalization models can make models less biased. We conclude that it is likely to be impossible to eliminate all training bias without an unlimited size dataset, and our experiments show that the bias can be reduced and quantified. We believe the next best to a perfect unbiased predictor is one that has minimized the remaining known bias.

show abstract