Photorealistic Image Synthesis for Object Instance Detection

Hodaň, Tomáš; Vineet, Vibhav; Gal, Ran; Shalev, Emanuel; Hanzelka, Jon; Connell, Treb; Urbina, Pedro; Sinha, Sudipta N.; Guenter, Brian

doi:10.1109/icip.2019.8803821

Cited by 109 publications

(68 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The first category is the model-based approach where a model is formulated to observe the data and a dedicated engine renders the data. This approach has been used for increasing the training dataset of urban driving environment [37,38], object detection [39], text segmentation [40], realistic digital brain-phantom generation [41], synthetic agar plate image generation [42]. Designing such specialized data generation engine requires accurate model and deep knowledge of the specific domain.…”

Section: Related Workmentioning

confidence: 99%

GAN-based synthetic brain PET image generation

2020

View full text Add to dashboard Cite

In recent days, deep learning technologies have achieved tremendous success in computer vision-related tasks with the help of large-scale annotated dataset. Obtaining such dataset for medical image analysis is very challenging. Working with the limited dataset and small amount of annotated samples makes it difficult to develop a robust automated disease diagnosis model. We propose a novel approach to generate synthetic medical images using generative adversarial networks (GANs). Our proposed model can create brain PET images for three different stages of Alzheimer's disease-normal control (NC), mild cognitive impairment (MCI), and Alzheimer's disease (AD).

show abstract

Section: Related Workmentioning

confidence: 99%

GAN-based synthetic brain PET image generation

2020

View full text Add to dashboard Cite

show abstract

“…More real‐time rendering generation pipelines have been developed in 3D development platforms utilizing non‐procedural physically based modelling [GWCV16, QY16, QZZ*17], non‐procedural non‐physically based modelling with domain and rendering randomization and object infusion [TPA*18] (Figure 12a) and procedural, physically based modelling in a structured domain and rendering randomization manner [PBB*18] (Figure 12b). Recently, offline rendering methods that employ both procedural and non‐procedural physically based modelling have been also introduced [WU18, TTB18, HVG*19].…”

Section: Image Synthesis Methods Overviewmentioning

confidence: 99%

“…[TPA*18], [WU18]

^{false(13 false)}

, [PBB*18], [KPL*19]

^{false(10 false)}

, [HCW19]

^{false(11 false)}

, [HVG*19], [ACF*19]

^{false(16 false)}

…”

Section: Introductionmentioning

confidence: 99%

A Survey of Image Synthesis Methods for Visual Machine Learning

Tsirikoglou

Eilertsen

Unger

2020

Computer Graphics Forum

View full text Add to dashboard Cite

Image synthesis designed for machine learning applications provides the means to efficiently generate large quantities of training data while controlling the generation process to provide the best distribution and content variety. With the demands of deep learning applications, synthetic data have the potential of becoming a vital component in the training pipeline. Over the last decade, a wide variety of training data generation methods has been demonstrated. The potential of future development calls to bring these together for comparison and categorization. This survey provides a comprehensive list of the existing image synthesis methods for visual machine learning. These are categorized in the context of image generation, using a taxonomy based on modelling and rendering, while a classification is also made concerning the computer vision applications they are used. We focus on the computer graphics aspects of the methods, to promote future image generation for machine learning. Finally, each method is assessed in terms of quality and reported performance, providing a hint on its expected learning potential. The report serves as a comprehensive reference, targeting both groups of the applications and data development sides. A list of all methods and papers reviewed herein can be found at https://computergraphics.on.liu.se/image_synthesis_methods_for_visual_machine_learning/.

show abstract

“…One of the most significant problem for using deep-learning skill in monocular 3D pose estimation is the deficiency of image dataset with accurate annotations of 3D pose information. Recently, researchers start using synthetic images dataset to train deep learning network for object detection [44] [45], key-points localization [46], semantic segmentation [47] [2] for flying machine pose estimation like ours, however, both their datasets are based on very few specific models with limited general applicability.…”

Section: Synthetic Image Datasetmentioning

confidence: 99%

2D-Key-Points-Localization-Driven 3D Aircraft Pose Estimation

Rong

Zhu

2020

IEEE Access

View full text Add to dashboard Cite

In this paper, we are interesting in inferring 3D pose estimation of aircraft object leveraging 2D key-points localization. Monocular vision based pose estimation for aircraft can be widely utilized in airspace tasks like flight control system, air traffic management, autonomous navigation and air defense system. Nonetheless, prior methods using directly regression or classification can not meet the requirements of high precision in aircraft pose estimation context, other approaches using PnP algorithms that need additional information such as template 3D model or depth as prior knowledge. These methods do not exploit to full advantage the correlation information between 2D key-points and 3D pose. In this paper, we present a multi-branch network, named AirPose network, using convolutional neural network to address 3D pose estimation based on 2D key-points information. In the meantime, a novel feature fusion method is explored to enable orientation estimation branch adequately exploit key-points information. Our feature fusion method significantly decreases 3D pose estimation error also avoids the involvement of RANSAC based PnP algorithms. To address the problem that there is no available dedicated aircraft 3D pose dataset for training and testing, we build a visual simulation platform on Unreal Engine 4 applying domain randomization (DR) skill, named AKO platform, which generates aircraft images automatically labeled with 3D orientation and key-points location. The dataset is called AKO dataset. We implement a series of ablation experiments to evaluate our framework for aircraft object detection, key-points localization and orientation estimation on AKO dataset. Experiments show that our proposed AirPose network leveraging AKO dataset can achieve convincing results for each of the tasks.

show abstract

Photorealistic Image Synthesis for Object Instance Detection

Cited by 109 publications

References 42 publications

GAN-based synthetic brain PET image generation

GAN-based synthetic brain PET image generation

A Survey of Image Synthesis Methods for Visual Machine Learning

2D-Key-Points-Localization-Driven 3D Aircraft Pose Estimation

Contact Info

Product

Resources

About