Deep intrinsic decomposition trained on surreal scenes yet with realistic light effects

Sial, Hassan Ahmed; Baldrich, Ramon Miró; Vanrell, María

doi:10.1364/josaa.37.000001

Cited by 7 publications

(10 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, Sial et al . [SBV20] used multi‐sided rooms with highly variable reflectances on the walls instead of environmental maps to illuminate non‐procedural and non‐physically based modelled 3D scenes, rendered in an object infusion, offline fashion (Figure 15c). They claim that the cast shadows and the physical consistency that some point light sources in a synthetic 3D textured room generate, can benefit the image decomposition task.…”

Section: Image Synthesis Methods Overviewmentioning

confidence: 99%

A Survey of Image Synthesis Methods for Visual Machine Learning

Tsirikoglou

Eilertsen

Unger

2020

Computer Graphics Forum

View full text Add to dashboard Cite

Image synthesis designed for machine learning applications provides the means to efficiently generate large quantities of training data while controlling the generation process to provide the best distribution and content variety. With the demands of deep learning applications, synthetic data have the potential of becoming a vital component in the training pipeline. Over the last decade, a wide variety of training data generation methods has been demonstrated. The potential of future development calls to bring these together for comparison and categorization. This survey provides a comprehensive list of the existing image synthesis methods for visual machine learning. These are categorized in the context of image generation, using a taxonomy based on modelling and rendering, while a classification is also made concerning the computer vision applications they are used. We focus on the computer graphics aspects of the methods, to promote future image generation for machine learning. Finally, each method is assessed in terms of quality and reported performance, providing a hint on its expected learning potential. The report serves as a comprehensive reference, targeting both groups of the applications and data development sides. A list of all methods and papers reviewed herein can be found at https://computergraphics.on.liu.se/image_synthesis_methods_for_visual_machine_learning/.

show abstract

Section: Image Synthesis Methods Overviewmentioning

confidence: 99%

A Survey of Image Synthesis Methods for Visual Machine Learning

Tsirikoglou

Eilertsen

Unger

2020

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…Camera design [33], [154], [153] Noise modeling [145], [3], [28] Intrinsic decomposition [23], [196], [148], [20] 16 [21], [266], [27], [225] Table 2.1: Computer vision applications that benefit from synthetic training data, along with associated image synthesis approaches (superscripts denote cross-application approach).…”

Section: Computational Photography and Image Formationmentioning

confidence: 99%

“…Large-scale synthetic data that come with pixel-perfect automatic or semi-automatic annotations was the perfect solution for tasks like these, when annotation of real captured data has been reported to need 20 to 90 minutes per frame [40,51]. Aside from recognition tasks, intrinsic decomposition [225,266], depth estimation [10,259], optical flow [41,60,161], and human body pose estimation [44,65,182,223] have also used image synthesis extensively. Following the literature, non-procedural physically-based modeling is a common choice when it comes to building virtual worlds while real-time and offline rendering are both 5.3 • Data generation 59 frequently used synthesis options.…”

Section: Computer Graphicsmentioning

confidence: 99%

Synthetic data for visual machine learning : A data-centric approach

Tsirikoglou

2022

Linköping Studies in Science and Technology. Dissertations

View full text Add to dashboard Cite

Deep learning allows computers to learn from observations, or else training data. Successful application development requires skills in neural network design, adequate computational resources, and a training data distribution that covers the application domain. We are currently witnessing an artificial intelligence (AI) outbreak with enough computational power to train very deep networks and build models that achieve similar or better than human performance. The crucial factor for the algorithms to succeed has proven to be the training data fed to the learning process. Too little or low quality or out-of-the-target distribution data will lead to poorly performing models no matter the capacity and the data regularization methods.This thesis is a data-centric approach to AI and presents a set of contributions related to synthesizing images for training supervised visual machine learning. It is motivated by the profound potential of synthetic data in cases of low availability of captured data, expensive acquisition and annotation, and privacy and ethical issues. The presented work aims to generate images similar to samples drawn from the target distribution and evaluate the generated data as the sole training data source and in conjunction with captured imagery. For this, two synthesis methods are explored: computer graphics and generative modeling. Computer graphics-based generation methods and synthetic datasets for computer vision tasks are thoroughly reviewed. In the same context, a system employing procedural modeling and physically-based rendering is introduced for data generation for urban scene understanding. The scheme is flexible, easily scalable, and produces complex and diverse images with pixel-perfect annotations at no cost. Generative Adversarial Networks (GANs) are also used to generate images for small data scenarios augmentation. The strategy advances the model's performance and robustness. Finally, ensembles of independently trained GANs investigate ways to improve images' diversity and create synthetic data to serve as the only training source.The application areas of the presented contributions relate to two image modalities, natural and histopathology images, to cover different aspects in the generation methods and the tasks' characteristics and requirements. There are showcased synthesized examples of natural images for automotive applications and weather classification, and histopathology images for breast cancer and colon adenocarcinoma metastasis detection. This thesis, as a whole, promotes data-centric supervised deep learning development by highlighting the potential of synthetic data as a training data resource. It emphasizes the control over the formation process, the ability of multi-modality formats, and the automatic generation of annotations.

show abstract

“…Baslamisli et al propose fine-grained shading decomposition [38]. We refer readers to the work of Sial et al, which provides a comprehensive overview of deep-learning-based methods and large-scale datasets [39].…”

Section: Related Workmentioning

confidence: 99%