Driving in the Matrix: Can Virtual Worlds Replace Human-Generated Annotations for Real World Tasks?

Johnson-Roberson, Matthew; Barto, Charles; Mehta, Rounak; Sridhar, Sharath Nittur; Rosaen, Karl; Vasudevan, Ram

doi:10.48550/arxiv.1610.01983

Cited by 62 publications

(93 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DeepGTA-PreSIL integrates DeepGTAV and GTAVisionExport [51], the technique presented in [17], to extract depth and stencil buffers from the rendering pipeline. With those and the world coordinates of objects extracted in GTAV, pixel-wise object segmentation data can be extracted.…”

Section: Methodsmentioning

confidence: 99%

“…5) Modifications to capture 4k image data (although we did not analyze the influence of high resolution data). In this work, those adaptations were used to capture object detection data from a UAV perspective (in comparison to an autonomous car scenario in previous works [14], [17]).…”

Section: Methodsmentioning

confidence: 99%

“…Instead of laboriously creating simulation engines, some researchers use the computer game GTAV [16] to generate data. With DeepGTAV [17], researchers leverage GTAV to gather data for autonomous driving. PreSIL [14] builds upon this approach to refine the data acquisition process with their tool DeepGTAV-PreSIL.…”

Section: A Related Workmentioning

confidence: 99%

“…It offers numerous detailed object models that interact in a large world with realistic graphics and physics simulations. Building on previous works [14], [17]- [19], we extend the DeepGTAV data extraction tool to work for airborne scenarios by extending its functionalities regarding in-game agent and camera position and rotation, environment manipulation, object spawning and metadata extraction.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Leveraging Synthetic Data in Object Detection on Unmanned Aerial Vehicles

Kiefer¹,

Ott²,

Zell³

2021

Preprint

View full text Add to dashboard Cite

Acquiring data to train deep learning-based object detectors on Unmanned Aerial Vehicles (UAVs) is expensive, time-consuming and may even be prohibited by law in specific environments. On the other hand, synthetic data is fast and cheap to access. In this work, we explore the potential use of synthetic data in object detection from UAVs across various application environments. For that, we extend the open-source framework DeepGTAV to work for UAV scenarios. We capture various largescale high-resolution synthetic data sets in several domains to demonstrate their use in real-world object detection from UAVs by analyzing multiple training strategies across several models. Furthermore, we analyze several different data generation and sampling parameters to provide actionable engineering advice for further scientific research. The DeepGTAV framework is available at https://git.io/Jyf5j.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: A Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Leveraging Synthetic Data in Object Detection on Unmanned Aerial Vehicles

Kiefer¹,

Ott²,

Zell³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Following six object detection datasets are used: Pascal VOC [11], Clipart [20], Comic [20], Sim10k [22], Cityscapes [9] and FoggyCityscapes [43]. Pascal VOC contains 20 categories of common realworld objects and [42], we evaluate the domain adaptation performance of different methods on the following four domain adaptation tasks, VOC-to-Clipart, VOC-to-Comic2k, Sim10k-to-Cityscapes, Cityscapes-to-FoggyCityscapes, and report the mean average precision (mAP) with a threshold of 0.5.…”

Section: Datasetsmentioning

confidence: 99%

Decoupled Adaptation for Cross-Domain Object Detection

Jiang¹,

Chen²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Cross-domain object detection is more challenging than object classification since multiple objects exist in an image and the location of each object is unknown in the unlabeled target domain. As a result, when we adapt features of different objects to enhance the transferability of the detector, the features of the foreground and the background are easy to be confused, which may hurt the discriminability of the detector. Besides, previous methods focused on category adaptation but ignored another important part for object detection, i.e., the adaptation on bounding box regression. To this end, we propose D-adapt, namely Decoupled Adaptation, to decouple the adversarial adaptation and the training of the detector. Besides, we fill the blank of regression domain adaptation in object detection by introducing a bounding box adaptor. Experiments show that D-adapt achieves state-of-theart results on four cross-domain object detection tasks and yields 17% and 21% relative improvement on benchmark datasets Clipart1k and Comic2k in particular.

show abstract

Attention-based domain adaptation for single-stage detectors

Vidit

Salzmann

2022

Machine Vision and Applications

View full text Add to dashboard Cite

While domain adaptation has been used to improve the performance of object detectors when the training and test data follow different distributions, previous work has mostly focused on two-stage detectors. This is because their use of region proposals makes it possible to perform local adaptation, which has been shown to significantly improve the adaptation effectiveness. Here, by contrast, we target single-stage architectures, which are better suited to resource-constrained detection than two-stage ones but do not provide region proposals. To nonetheless benefit from the strength of local adaptation, we introduce an attention mechanism that lets us identify the important regions on which adaptation should focus. Our method gradually adapts the features from global, image level to local, instance level. Our approach is generic and can be integrated into any Single-Shot Detector. We demonstrate this on standard benchmark datasets by applying it to both the single-shot detector (SSD) and a recent variant of the You Only Look Once detector (YOLOv5). Furthermore, for equivalent single-stage architectures, our method outperforms the state-of-the-art domain adaptation techniques even though they were designed for specific detectors.

show abstract

Driving in the Matrix: Can Virtual Worlds Replace Human-Generated Annotations for Real World Tasks?

Cited by 62 publications

References 0 publications

Leveraging Synthetic Data in Object Detection on Unmanned Aerial Vehicles

Leveraging Synthetic Data in Object Detection on Unmanned Aerial Vehicles

Decoupled Adaptation for Cross-Domain Object Detection

Attention-based domain adaptation for single-stage detectors

Contact Info

Product

Resources

About