Image-To-Image Translation Using a Cross-Domain Auto-Encoder and Decoder

Abstract

Recently, several studies have focused on image-to-image translation. However, the quality of the translation results is lacking in certain respects. We propose a new image-to-image translation method to minimize such shortcomings using an auto-encoder and an auto-decoder. This method includes pre-training two auto-encoders and decoder pairs for each source and target image domain, cross-connecting two pairs and adding a feature mapping layer. Our method is quite simple and straightforward to adopt but very effective in practice, and we experimentally demonstrated that our method can significantly enhance the quality of image-to-image translation. We used the well-known cityscapes, horse2zebra, cat2dog, maps, summer2winter, and night2day datasets. Our method shows qualitative and quantitative improvements over existing models. Appl. Sci. 2019, 9, 4780 2 of 16 translation method. In this paper, we propose a new translation method where the mapping process from source domain to target domain can work better, and thus suppress mistranslation by transferring the learning of domain and target feature spaces using both an auto-encoder and decoder. In contrast to commonly-used feature space learning methods [9][10][11][12][13], our approach utilizes an auto-encoder and an auto-decoder-which are pretrained for the source domain and target domain, respectively-and cross-connects them by mapping their feature spaces across the two domains. Appl. Sci. 2019, 9, x FOR PEER REVIEW 2 of 17 Author Contributions: Conceptualization and methodology, J.Y., H.E., and Y.S.C.; investigation, J.Y. and H.E.; data curation and conceiving experiments, J.Y.; performing experiments and designing results, H.E.; writing-original draft preparation, J.Y. and H.E.; writing-review and editing, J.Y., H.E., and Y.S.C. Appendix AWe provide our source code at https://github.com/Rooooss/I2I_CD_AE.data curation and conceiving experiments, J.Y.; performing experiments and designing results, H.E.; writingoriginal draft preparation, J.Y. and H.E.; writing-review and editing, J.Y., H.E., and Y.S.C.

Reusing Monolingual Pre-Trained Models by Cross-Connecting Seq2seq Models for Machine Translation

“…In the field of computer vision, Ref. [36] shares similar concept with ours for the image translation task.…”

“…The additional parameters help the components to adapt to each other. This is similar to the 'feature mapping layer' from [36] that fills the gap between the representations, which the encoder is pre-trained to generate and the decoder is pre-trained to reconstruct from. It can be said that this layer maps the separately pre-trained language spaces.…”

GlauCUTU: Time Until Perceived Virtual Reality Perimetry With Humphrey Field Analyzer Prediction-Based Artificial Intelligence

“…Moreover, we also incorporated an autoencoder model [42] into the trained network. The autoencoder is an unsupervised learning technique that consists of two networks, encoder and decoder.…”

Scale-adaptive model for detection and grading of age-related macular degeneration from color retinal fundus images

“…Auto-encoders may be used for data compression 35 in which the compressed representation is used to keep the information in a more compact format, as well as denoising, in which the model is trained to reconstruct clean data from noisy input. It can also be used for image-to-image translation 36 by randomly sampling from the compressed representation and decoding it to generate a new image as well as dimensionality reduction 37 – 40 by training an auto-encoder, the network can learn a compressed representation of the data that captures the most important features and capable of generating new images such as variational auto-encoder (VAE) 41 . In the auto-encoder-based model, the latent space layer is responsible for performing dimensionality reduction.…”