Landmark Assisted CycleGAN for Cartoon Face Generation

Wu, Ruizheng; Gu, Xiaodong; Tao, Xin; Shen, Xiaoyong; Tai, Yu-Wing; Jia, J iaya

doi:10.48550/arxiv.1907.01424

Cited by 4 publications

(5 citation statements)

References 32 publications

(73 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A dataset named "Urban100" was used to train and test my model of super resolution. Also a pre-trained files from the dlib called "shape_predictor_68_face _landmarks.dat" which uses the method of landmarks assisted CycleGAN where face landmarks of real and cartoon faces are used in conjunction with the original images of real and cartoon faces [8] to extract the specific 68 feature points of the human faces. My experiment is based on python 3.9.…”

Section: Environment and Toolsmentioning

confidence: 99%

Improving the effect of low-resolution face images output in AnimeGAN

2023

Eighth International Conference on Electronic Technology and Information Science (ICETIS 2023)

View full text Add to dashboard Cite

In this paper, a novel approach for improving the output cartoon-style effect of low-resolution face images in AnimeGAN is proposed, which is an useful and effective method to generate high-quality cartoon-style face images. This new approach I proposed combines generative adversarial networks (GAN), AnimeGAN and SRGAN. The previous existing methods do not give satisfactory results on processing low-resolution images. The cartoon-style images generated from the LR images have many significant visual issues. For example, the output cartoon-style faces have some unreasonable and weird shadows and wrinkles, which is unreal and far from the effect of original images. In this paper, I introduce a new method of using SRGAN to increase the resolution of the original LR images in order to improve the output effect in AnimeGAN. At last, the experimental results show that my combined method has good output from LR images and improves the cartoon-style faces effect as well as the performance of AnimeGAN.

show abstract

Section: Environment and Toolsmentioning

confidence: 99%

Improving the effect of low-resolution face images output in AnimeGAN

2023

Eighth International Conference on Electronic Technology and Information Science (ICETIS 2023)

View full text Add to dashboard Cite

show abstract

“…This method has been widely used in tasks such as colorization, super-resolution, and style transfer. Based on CycleGAN, different models have been proposed for face transfer [25], Chinese handwritten character generation [26], image generation from text [27], image correction [28], and tasks in the audio field [29][30][31].…”

Section: Cycle-consistent Adversarial Networkmentioning

confidence: 99%

T5-Based Model for Abstractive Summarization: A Semi-Supervised Learning Approach with Consistency Loss Functions

Wang

Xie

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

Text summarization is a prominent task in natural language processing (NLP) that condenses lengthy texts into concise summaries. Despite the success of existing supervised models, they often rely on datasets of well-constructed text pairs, which can be insufficient for languages with limited annotated data, such as Chinese. To address this issue, we propose a semi-supervised learning method for text summarization. Our method is inspired by the cycle-consistent adversarial network (CycleGAN) and considers text summarization as a style transfer task. The model is trained by using a similar procedure and loss function to those of CycleGAN and learns to transfer the style of a document to its summary and vice versa. Our method can be applied to multiple languages, but this paper focuses on its performance on Chinese documents. We trained a T5-based model and evaluated it on two datasets, CSL and LCSTS, and the results demonstrate the effectiveness of the proposed method.

show abstract

“…Here, a single detector is used across both the domains and is trained together with adversarial networks. Wu et al [14] use facial landmarks, with a landmark consistency loss, to produce structure-preserving facial transformations. The facial landmarks, are concatenated with the image from the source domain, and provided to the generator.…”

Section: Task-integrated Image-to-image Translationmentioning

confidence: 99%

“…Several approaches have been proposed to improve the quality of image-to-image translation, e.g. by integrating information from auxiliary tasks such as object detection [11]- [13], facial landmark detection [14], [15], or semantic segmentation [16], [17]. However, some of these methods are ill-suited to address the problem in the endoscopic domain, in particular for mitral valve repair.…”

Section: Introductionmentioning

confidence: 99%

Mutually improved endoscopic image synthesis and landmark detection in unpaired image-to-image translation

Sharan,

Romano,

Koehler

et al. 2021

Preprint

View full text Add to dashboard Cite

The CycleGAN framework allows for unsupervised image-to-image translation of unpaired data. In a scenario of surgical training on a physical surgical simulator, this method can be used to transform endoscopic images of phantoms into images which more closely resemble the intra-operative appearance of the same surgical target structure. This can be viewed as a novel augmented reality approach, which we coined Hyperrealism in previous work. In this use case, it is of paramount importance to display objects like needles, sutures or instruments consistent in both domains while altering the style to a more tissue-like appearance. Segmentation of these objects would allow for a direct transfer, however, contouring of these, partly tiny and thin foreground objects is cumbersome and perhaps inaccurate. Instead, we propose to use landmark detection on the points when sutures pass into the tissue. This objective is directly incorporated into a CycleGAN framework by treating the performance of pre-trained detector models as an additional optimization goal. We show that a task defined on these sparse landmark labels improves consistency of synthesis by the generator network in both domains. Comparing a baseline CycleGAN architecture to our proposed extension (DetCycleGAN), mean precision (PPV) improved by + 61.32, mean sensitivity (TPR) by +37.91, and mean F 1 score by +0.4743. Furthermore, it could be shown that by dataset fusion, generated intraoperative images can be leveraged as additional training data for the detection network itself. The data is released within the scope of the AdaptOR MICCAI Challenge 2021 at https://adaptor2021.github.io/, and code at https: //github.com/Cardio-AI/detcyclegan_pytorch.

show abstract

Landmark Assisted CycleGAN for Cartoon Face Generation

Cited by 4 publications

References 32 publications

Improving the effect of low-resolution face images output in AnimeGAN

Improving the effect of low-resolution face images output in AnimeGAN

T5-Based Model for Abstractive Summarization: A Semi-Supervised Learning Approach with Consistency Loss Functions

Mutually improved endoscopic image synthesis and landmark detection in unpaired image-to-image translation

Contact Info

Product

Resources

About