Deep CG2Real: Synthetic-to-Real Translation via Image Disentanglement

Bi, Sai; Sunkavalli, Kalyan; Perazzi, Federico; Shechtman, Eli; Kim, Vladimir; Ramamoorthi, Ravi

doi:10.1109/iccv.2019.00282

Cited by 38 publications

(26 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The image classification is used for the DI-HV. CG Real dataset is proposed by [3], which translates between synthetic indoor images and real indoor scenes. Semantic scene parsing is used as the DI-HV, where we generate the pseudo semantic segmentation mask by HRNet [49].…”

Section: Methodsmentioning

confidence: 99%

Separating Content and Style for Unsupervised Image-to-Image Translation

Liu¹,

Wang²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Unsupervised image-to-image translation aims to learn the mapping between two visual domains with unpaired samples. Existing works focus on disentangling domaininvariant content code and domain-specific style code individually for multimodal purposes. However, less attention has been paid to interpreting and manipulating the translated image. In this paper, we propose to separate the content code and style code simultaneously in a unified framework. Based on the correlation between the latent features and the high-level domain-invariant tasks, the proposed framework demonstrates superior performance in multimodal translation, interpretability and manipulation of the translated image. Experimental results show that the proposed approach outperforms the existing unsupervised image translation methods in terms of visual quality and diversity. Code and data have been released at https://github.com/DreamtaleCore/SCS-UIT.

show abstract

Section: Methodsmentioning

confidence: 99%

Separating Content and Style for Unsupervised Image-to-Image Translation

Liu¹,

Wang²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Early approaches used simple hand‐modelled scenes, populated with either single or few objects, with accurate light transport simulation enabled by photon mapping [BSvdW*13]. Later methods built their scene content using 3D models and scene databases, along with rendering randomization, measured materials and environment maps for global illumination, while employing various flavours of path‐tracing and tone mapping algorithms [RRF*16, LS18, BLG18, BSP*19] (Figure 15a). In the same spirit, Baslamisli et al .…”

Section: Image Synthesis Methods Overviewmentioning

confidence: 99%

A Survey of Image Synthesis Methods for Visual Machine Learning

Tsirikoglou

Eilertsen

Unger

2020

Computer Graphics Forum

View full text Add to dashboard Cite

Image synthesis designed for machine learning applications provides the means to efficiently generate large quantities of training data while controlling the generation process to provide the best distribution and content variety. With the demands of deep learning applications, synthetic data have the potential of becoming a vital component in the training pipeline. Over the last decade, a wide variety of training data generation methods has been demonstrated. The potential of future development calls to bring these together for comparison and categorization. This survey provides a comprehensive list of the existing image synthesis methods for visual machine learning. These are categorized in the context of image generation, using a taxonomy based on modelling and rendering, while a classification is also made concerning the computer vision applications they are used. We focus on the computer graphics aspects of the methods, to promote future image generation for machine learning. Finally, each method is assessed in terms of quality and reported performance, providing a hint on its expected learning potential. The report serves as a comprehensive reference, targeting both groups of the applications and data development sides. A list of all methods and papers reviewed herein can be found at https://computergraphics.on.liu.se/image_synthesis_methods_for_visual_machine_learning/.

show abstract

“…Camera design [33], [154], [153] Noise modeling [145], [3], [28] Intrinsic decomposition [23], [196], [148], [20] 16 [21], [266], [27], [225] Table 2.1: Computer vision applications that benefit from synthetic training data, along with associated image synthesis approaches (superscripts denote cross-application approach).…”

Section: Computational Photography and Image Formationmentioning

confidence: 99%

Synthetic data for visual machine learning : A data-centric approach

Tsirikoglou

2022

Linköping Studies in Science and Technology. Dissertations

View full text Add to dashboard Cite

Deep learning allows computers to learn from observations, or else training data. Successful application development requires skills in neural network design, adequate computational resources, and a training data distribution that covers the application domain. We are currently witnessing an artificial intelligence (AI) outbreak with enough computational power to train very deep networks and build models that achieve similar or better than human performance. The crucial factor for the algorithms to succeed has proven to be the training data fed to the learning process. Too little or low quality or out-of-the-target distribution data will lead to poorly performing models no matter the capacity and the data regularization methods.This thesis is a data-centric approach to AI and presents a set of contributions related to synthesizing images for training supervised visual machine learning. It is motivated by the profound potential of synthetic data in cases of low availability of captured data, expensive acquisition and annotation, and privacy and ethical issues. The presented work aims to generate images similar to samples drawn from the target distribution and evaluate the generated data as the sole training data source and in conjunction with captured imagery. For this, two synthesis methods are explored: computer graphics and generative modeling. Computer graphics-based generation methods and synthetic datasets for computer vision tasks are thoroughly reviewed. In the same context, a system employing procedural modeling and physically-based rendering is introduced for data generation for urban scene understanding. The scheme is flexible, easily scalable, and produces complex and diverse images with pixel-perfect annotations at no cost. Generative Adversarial Networks (GANs) are also used to generate images for small data scenarios augmentation. The strategy advances the model's performance and robustness. Finally, ensembles of independently trained GANs investigate ways to improve images' diversity and create synthetic data to serve as the only training source.The application areas of the presented contributions relate to two image modalities, natural and histopathology images, to cover different aspects in the generation methods and the tasks' characteristics and requirements. There are showcased synthesized examples of natural images for automotive applications and weather classification, and histopathology images for breast cancer and colon adenocarcinoma metastasis detection. This thesis, as a whole, promotes data-centric supervised deep learning development by highlighting the potential of synthetic data as a training data resource. It emphasizes the control over the formation process, the ability of multi-modality formats, and the automatic generation of annotations.

show abstract

Deep CG2Real: Synthetic-to-Real Translation via Image Disentanglement

Cited by 38 publications

References 31 publications

Separating Content and Style for Unsupervised Image-to-Image Translation

Separating Content and Style for Unsupervised Image-to-Image Translation

A Survey of Image Synthesis Methods for Visual Machine Learning

Synthetic data for visual machine learning : A data-centric approach

Contact Info

Product

Resources

About