DualGAN: Unsupervised Dual Learning for Image-to-Image Translation

Yi, Zili; Zhang, Hao; Tan, Ping; Gong, Minglun

doi:10.1109/iccv.2017.310

Cited by 1,954 publications

(1,412 citation statements)

References 23 publications

Supporting

Mentioning

1,339

Contrasting

Unclassified

Order By: Relevance

“…The MMD term in our formulation only ensures that the two distributions agree globally in the latent space, whereas both JLMA and GUMA have a term that ensures, for each instance, that the local geometry is preserved between domains. This is the difference between manifold superimposing [18,17,8] and manifold alignment discussed in the MAGAN paper [1]. Furthermore, GUMA's assumption that individual cells can be matched 1-to-1 between the two input domains is not generally true, most obviously when n 1 = n 2 .…”

Section: Comparison Of These Three Algorithms With Our Algorithmmentioning

confidence: 99%

Jointly embedding multiple single-cell omics measurements

Liu

Huang

Singh

et al. 2019

Preprint

102

View full text Add to dashboard Cite

Many single-cell sequencing technologies are now available, but it is still difficult to apply multiple sequencing technologies to the same single cell. In this paper, we propose an unsupervised manifold alignment algorithm, MMD-MA, for integrating multiple measurements carried out on disjoint aliquots of a given population of cells. Effectively, MMD-MA performs an in silico co-assay by embedding cells measured in different ways into a learned latent space. In the MMD-MA algorithm, single-cell data points from multiple domains are aligned by optimizing an objective function with three components: (1) a maximum mean discrepancy (MMD) term to encourage the differently measured points to have similar distributions in the latent space, (2) a distortion term to preserve the structure of the data between the input space and the latent space, and (3) a penalty term to avoid collapse to a trivial solution. Notably, MMD-MA does not require any correspondence information across data modalities, either between the cells or between the features. Furthermore, MMD-MA's weak distributional requirements for the domains to be aligned allow the algorithm to integrate heterogeneous types of single cell measures, such as gene expression, DNA accessibility, chromatin organization, methylation, and imaging data. We demonstrate the utility of MMD-MA in simulation experiments and using a real data set involving single-cell gene expression and methylation data.

show abstract

Section: Comparison Of These Three Algorithms With Our Algorithmmentioning

confidence: 99%

Jointly embedding multiple single-cell omics measurements

Liu

Huang

Singh

et al. 2019

Preprint

102

View full text Add to dashboard Cite

show abstract

“…However, this method relied on paired data for supervised learning. To avoid this prerequisite, CycleGAN, DiscoGAN, and DualGAN were designed following the cycle consistency for training using unpaired data. These series of methods have been proven effective in various tasks, such as collection style transfer, object transfiguration, season transfer, and generating photographs from sketches .…”

Section: Related Workmentioning

confidence: 99%

Multitask learning on monocular water images: Surface reconstruction and image synthesis

Xie

Zhai

Hou

et al. 2019

Computer Animation & Virtual

View full text Add to dashboard Cite

In this paper, we present a new strategy, a joint deep learning architecture, for two classic tasks in computer graphics: water surface reconstruction and water image synthesis. Modeling water surfaces from single images can be regarded as the inverse of image rendering, which converts surface geometries into photorealistic images. On the basis of this fact, we therefore consider these two problems as a cycle image‐to‐image translation and propose to tackle them together using a pair of neural networks, with the three‐dimensional surface geometries being represented as two‐dimensional surface normal maps. Furthermore, we also estimate the imaging parameters from the existing water images with a subnetwork to reuse the lighting conditions when synthesizing new images. Experiments demonstrate that our method achieves an accurate reconstruction of surfaces from monocular images efficiently and produces visually plausible new images under variable lighting conditions.

show abstract

“…In particular, without paired training samples, the original GANs cannot guarantee that the output imitations contain the same semantic information as that of the input images. Cycle-GAN [14], DiscoGAN [15], DualGAN [16] proposed cycleconsistent adversarial networks to address the unpaired imageto-image translation problem. They simultaneously trained two pairs of generative networks and discriminative networks, one to produce imitative paintings and the other to transform the imitation back to the original photograph and pursue cycle consistency.…”

mentioning

confidence: 99%

“…Considering the wide application of style transfer on mobile devices, space-saving is an important algorithm design consideration. Methods of CycleGAN [14], DiscoGAN [15], DualGAN [16] could only transfer one style per network. In this work, we propose a gated transformer module to achieve multi-collection style transfer in a single network.…”

mentioning

confidence: 99%

Gated-GAN: Adversarial Gated Networks for Multi-Collection Style Transfer

Chen

Yang

et al. 2019

IEEE Trans. on Image Process.

125

View full text Add to dashboard Cite

Style transfer describes the rendering of an image's semantic content as different artistic styles. Recently, generative adversarial networks (GANs) have emerged as an effective approach in style transfer by adversarially training the generator to synthesize convincing counterfeits. However, traditional GAN suffers from the mode collapse issue, resulting in unstable training and making style transfer quality difficult to guarantee. In addition, the GAN generator is only compatible with one style, so a series of GANs must be trained to provide users with choices to transfer more than one kind of style. In this paper, we focus on tackling these challenges and limitations to improve style transfer. We propose adversarial gated networks (Gated-GAN) to transfer multiple styles in a single model. The generative networks have three modules: an encoder, a gated transformer, and a decoder. Different styles can be achieved by passing input images through different branches of the gated transformer. To stabilize training, the encoder and decoder are combined as an auto-encoder to reconstruct the input images. The discriminative networks are used to distinguish whether the input image is a stylized or genuine image. An auxiliary classifier is used to recognize the style categories of transferred images, thereby helping the generative networks generate images in multiple styles. In addition, Gated-GAN makes it possible to explore a new style by investigating styles learned from artists or genres. Our extensive experiments demonstrate the stability and effectiveness of the proposed model for multi-style transfer.

show abstract

DualGAN: Unsupervised Dual Learning for Image-to-Image Translation

Cited by 1,954 publications

References 23 publications

Jointly embedding multiple single-cell omics measurements

Jointly embedding multiple single-cell omics measurements

Multitask learning on monocular water images: Surface reconstruction and image synthesis

Gated-GAN: Adversarial Gated Networks for Multi-Collection Style Transfer

Contact Info

Product

Resources

About