Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans

Eftekhar, Ainaz; Sax, Alexander F.; Bachmann, Roman; Malik, Jitendra; Zamir, Amir

doi:10.48550/arxiv.2110.04994

Cited by 1 publication

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[56] creates high quality room scenes, but has many manual steps including pose alignment and material assignment. [27] also utilizes 3D scans, and provides control over a wide range of scene parameters, including camera position, field of view, and lighting, as well as a number of per frame image cues. While these approaches produce high quality data for a particular captured scene, the pipeline still relies on 3D scans of the full scene, which imposes a bottleneck for scaling.…”

Section: Related Workmentioning

confidence: 99%

“…Rendering GI Physics Scaling DL Playing4Data [78] (Game) × (Game) × × UnrealCV [75] UE4 × UE4 × TDW [34] Unity × PhysX × iGibson [106] PyRender × PyBullet × Habitat [91] Magnum × Bullet × OpenRooms [56] OptiX -× × Omnidata [27] Blender -× Blenderproc [23] Blender Bullet × × Kubric Blender PyBullet Table 1. Rendering: Blender any OptiX are ray tracing engines, all others are based on rasterization; GI: support for global illumination; Physics: engine for physics simulation; Scaling: Easy to scale to very large datasets.…”

Section: Namementioning

confidence: 99%

See 1 more Smart Citation

Kubric: A scalable dataset generator

Greff¹,

Belletti²,

Beyer³

et al. 2022

Preprint

View full text Add to dashboard Cite

Data is the driving force of machine learning, with the amount and quality of training data often being more important for the performance of a system than architecture and training details. But collecting, processing and annotating real data at scale is difficult, expensive, and frequently raises additional privacy, fairness and legal concerns. Synthetic data is a powerful tool with the potential to address these shortcomings: 1) it is cheap 2) supports rich ground-truth annotations 3) offers full control over data and 4) can circumvent or mitigate problems regarding bias, privacy and licensing. Unfortunately, software tools for effective data generation are less mature than those for architecture design and training, which leads to fragmented generation efforts. To address these problems we introduce Kubric, an open-source Python framework that interfaces with PyBullet and Blender to generate photo-realistic scenes, with rich annotations, and seamlessly scales to large jobs distributed over thousands of machines, and generating TBs of data. We demonstrate the effectiveness of Kubric by presenting a series of 13 different generated datasets for tasks ranging from studying 3D NeRF models to optical flow estimation. We release Kubric, the used assets, all of the generation code, as well as the rendered datasets for reuse and modification. segmentation depth optical flow surface normals object coordinatesFigure 1. Example scene created and rendered with Kubric along with some of the automatically generated annotations.

show abstract

Section: Related Workmentioning

confidence: 99%