XGAN: Unsupervised Image-to-Image Translation for Many-to-Many Mappings

Royer, Amélie; Bousmalis, Konstantinos; Gouws, Stephan; Bertsch, Fred; Mosseri, Inbar; Cole, Forrester; Murphy, Kevin M.

doi:10.48550/arxiv.1711.05139

Cited by 25 publications

(48 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Image-to-image translation (I2I) [30,34,43,49,51,53,57,59,60] aims to transfer images from source to target domain with the content information preserved. Earlier methods [51,23,5,41] apply an adversarial loss [14], along with a reconstruction loss to train their model based on the paired training data.…”

Section: Image-to-image Translationmentioning

confidence: 99%

Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translation

Wang¹,

Zhou²,

Bao³

et al. 2021

Preprint

View full text Add to dashboard Cite

Contrastive learning shows great potential in unpaired image-to-image translation, but sometimes the translated results are in poor quality and the contents are not preserved consistently. In this paper, we uncover that the negative examples play a critical role in the performance of contrastive learning for image translation. The negative examples in previous methods are randomly sampled from the patches of different positions in the source image, which are not effective to push the positive examples close to the query examples. To address this issue, we present instance-wise hard Negative Example Generation for Contrastive learning in Unpaired image-to-image Translation (NEGCUT). Specifically, we train a generator to produce negative examples online. The generator is novel from two perspectives: 1) it is instance-wise which means that the generated examples are based on the input image, and 2) it can generate hard negative examples since it is trained with an adversarial loss. With the generator, the performance of unpaired image-to-image translation is significantly improved. Experiments on three benchmark datasets demonstrate that the proposed NEGCUT framework achieves state-of-the-art performance compared to previous methods.

show abstract

Section: Image-to-image Translationmentioning

confidence: 99%

Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translation

Wang¹,

Zhou²,

Bao³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…DiscoGAN [22] explores a few forms of cycle-consistency loss. XGAN [23] applies semantic consistency to embedded features. While CycleGAN achieves approximately deterministic mappings between domains, Augmented CycleGAN [24] extends the idea of CycleGAN by introducing stochastic many-to-many mappings.…”

Section: B Two-way Adversarial Networkmentioning

confidence: 99%

Galaxy Image Translation with Semi-supervised Noise-reconstructed Generative Adversarial Networks

Lin,

Fouchez,

Pasquet

2021

Preprint

View full text Add to dashboard Cite

Image-to-image translation with Deep Learning neural networks, particularly with Generative Adversarial Networks (GANs), is one of the most powerful methods for simulating astronomical images. However, current work is limited to utilizing paired images with supervised translation, and there has been rare discussion on reconstructing noise background that encodes instrumental and observational effects. These limitations might be harmful for subsequent scientific applications in astrophysics. Therefore, we aim to develop methods for using unpaired images and preserving noise characteristics in image translation. In this work, we propose a two-way image translation model using GANs that exploits both paired and unpaired images in a semisupervised manner, and introduce a noise emulating module that is able to learn and reconstruct noise characterized by high-frequency features. By experimenting on multi-band galaxy images from the Sloan Digital Sky Survey (SDSS) and the Canada France Hawaii Telescope Legacy Survey (CFHT), we show that our method recovers global and local properties effectively and outperforms benchmark image translation models. To our best knowledge, this work is the first attempt to apply semi-supervised methods and noise reconstruction techniques in astrophysical studies.

show abstract

“…This approach has a strong assumption that different domains should share the same low-dimensional representation in the network. XGAN [30] shares similar structure with UNIT [21] and it introduced the semantic consistency component in feature-level contrary to previous work of using pixel-level consistency. With a single auto-encoder to learn a common representation of different domains, DTN [33,36] transformed images in domains.…”

Section: Image-to-image Translationmentioning

confidence: 99%

Landmark Assisted CycleGAN for Cartoon Face Generation

Wu,

Gu,

Tao

et al. 2019

Preprint

View full text Add to dashboard Cite

In this paper, we are interested in generating an cartoon face of a person by using unpaired training data between real faces and cartoon ones. A major challenge of this task is that the structures of real and cartoon faces are in two different domains, whose appearance differs greatly from each other. Without explicit correspondence, it is difficult to generate a high quality cartoon face that captures the essential facial features of a person. In order to solve this problem, we propose landmark assisted CycleGAN, which utilizes face landmarks to define landmark consistency loss and to guide the training of local discriminator in CycleGAN. To enforce structural consistency in landmarks, we utilize the conditional generator and discriminator. Our approach is capable to generate high-quality cartoon faces even indistinguishable from those drawn by artists and largely improves state-of-the-art.

show abstract

XGAN: Unsupervised Image-to-Image Translation for Many-to-Many Mappings

Cited by 25 publications

References 0 publications

Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translation

Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translation

Galaxy Image Translation with Semi-supervised Noise-reconstructed Generative Adversarial Networks

Landmark Assisted CycleGAN for Cartoon Face Generation

Contact Info

Product

Resources

About