Kubric: A scalable dataset generator

Greff, Klaus; Belletti, Francois; Beyer, Lucas; Doersch, Carl; Du, Yilun; Duckworth, Daniel; Fleet, David J.; Gnanapragasam, Dan; Golemo, Florian; Herrmann, Charles; Kipf, Thomas; Kundu, Anjan; Lagun, Dmitry; Laradji, Issam; Liu, Hsueh-Ti; Meyer, Henning; Miao, Yishu; Nowrouzezahrai, Derek; Öztireli, Cengiz; Pot, Etienne; Radwan, Noha; Rebain, Daniel; Sabour, Sara; Sajjadi, Mehdi S. M.; Sela, Matan; Sitzmann, Vincent; Stone, Austin; Sun, Deqing; Vora, Suhani; Wang, Ziyu; Wu, Tianhao; Yi, Kwang Moo; Zhong, Fangcheng; Tagliasacchi, Andrea

doi:10.1109/cvpr52688.2022.00373

Cited by 122 publications

(33 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This was shown in [25] for close baseline setups, proposing a multi-view semantic fusion scheme from up to 8 input views onto a new virtual zenithal view. Concurrent to our work, [26] presents Kubric, a new flexible dataset generation framework with multiple dense annotation features.…”

Section: Introductionmentioning

confidence: 91%

MVMO: A Multi-Object Dataset for Wide Baseline Multi-View Semantic Segmentation

Alvarez-Gila

Weijer

Wang

et al. 2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

We present MVMO (Multi-View, Multi-Object dataset): a synthetic dataset of 116,000 scenes containing randomly placed objects of 10 distinct classes and captured from 25 camera locations in the upper hemisphere. MVMO comprises photorealistic, path-traced image renders, together with semantic segmentation ground truth for every view. Unlike existing multi-view datasets, MVMO features wide baselines between cameras and high density of objects, which lead to large disparities, heavy occlusions and view-dependent object appearance. Single view semantic segmentation is hindered by self and inter-object occlusions that could benefit from additional viewpoints. Therefore, we expect that MVMO will propel research in multi-view semantic segmentation and cross-view semantic transfer. We also provide baselines that show that new research is needed in such fields to exploit the complementary information of multi-view setups 1 .

show abstract

Section: Introductionmentioning

confidence: 91%

MVMO: A Multi-Object Dataset for Wide Baseline Multi-View Semantic Segmentation

Alvarez-Gila

Weijer

Wang

et al. 2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

show abstract

“…The function sequence of the simulation and rendering method can be implemented with programs such as Unreal Engine, Unity, and Blender. Frameworks such as BlenderProc [ 18 ] and Kubric [ 19 ] exist to support the generation of realistically looking images in these programs.…”

Section: State Of the Artmentioning

confidence: 99%

Automatic Image Generation Pipeline for Instance Segmentation of Deformable Linear Objects

Dirr

Gebauer

Yao

et al. 2023

Sensors

View full text Add to dashboard Cite

Robust detection of deformable linear objects (DLOs) is a crucial challenge for the automation of handling and assembly of cables and hoses. The lack of training data is a limiting factor for deep-learning-based detection of DLOs. In this context, we propose an automatic image generation pipeline for instance segmentation of DLOs. In this pipeline, a user can set boundary conditions to generate training data for industrial applications automatically. A comparison of different replication types of DLOs shows that modeling DLOs as rigid bodies with versatile deformations is most effective. Further, reference scenarios for the arrangement of DLOs are defined to generate scenes in a simulation automatically. This allows the pipelines to be quickly transferred to new applications. The validation of models trained with synthetic images and tested on real-world images shows the feasibility of the proposed data generation approach for segmentation of DLOs. Finally, we show that the pipeline yields results comparable to the state of the art but has advantages in reduced manual effort and transferability to new use cases.

show abstract

“…However, it uses the Unity Asset Store as the database, the quality of assets is not ensured, and manually annotating assets is still required. The concurrent work Kubric [GBB*22] mostly relies on existing public 3D assets. Instead, there is a high‐quality large‐scale commercial scene database behind our system for users to access as an online service.…”

Section: Related Workmentioning

confidence: 99%

“…Most recently, several systems [DSW*19, MTL*21, GSA*21, ESMZ21, BCD*21, GBB*22] are proposed to generate synthetic dataset from collections of 3D assets. Though the overall goal is similar, our system differs from them in several aspects.…”

Section: Related Workmentioning

confidence: 99%

“…BlenderProc [DSW*19] provides a configurable system that can generate photorealistic large‐scale imagery datasets from public 3D scene databases, but it has limited scriptable interfaces. NviSII [MTL*21], ThreeDWorld [GSA*21] and Kubric [GBB*22] uses python API to address this problem. It enables fine‐grained control of the scene, but it brings a tedious coding process lacking some high‐level built‐in functions for randomization.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

MINERVAS: Massive INterior EnviRonments VirtuAl Synthesis

Ren

Zhang

Zheng³

et al. 2022

Computer Graphics Forum

View full text Add to dashboard Cite

With the rapid development of data‐driven techniques, data has played an essential role in various computer vision tasks. Many realistic and synthetic datasets have been proposed to address different problems. However, there are lots of unresolved challenges: (1) the creation of dataset is usually a tedious process with manual annotations, (2) most datasets are only designed for a single specific task, (3) the modification or randomization of the 3D scene is difficult, and (4) the release of commercial 3D data may encounter copyright issue. This paper presents MINERVAS, a Massive INterior EnviRonments VirtuAl Synthesis system, to facilitate the 3D scene modification and the 2D image synthesis for various vision tasks. In particular, we design a programmable pipeline with Domain‐Specific Language, allowing users to select scenes from the commercial indoor scene database, synthesize scenes for different tasks with customized rules, and render various types of imagery data, such as color images, geometric structures, semantic labels. Our system eases the difficulty of customizing massive scenes for different tasks and relieves users from manipulating fine‐grained scene configurations by providing user‐controllable randomness using multilevel samplers. Most importantly, it empowers users to access commercial scene databases with millions of indoor scenes and protects the copyright of core data assets, e.g., 3D CAD models. We demonstrate the validity and flexibility of our system by using our synthesized data to improve the performance on different kinds of computer vision tasks. The project page is at https://coohom.github.io/MINERVAS.

show abstract

Kubric: A scalable dataset generator

Cited by 122 publications

References 65 publications

MVMO: A Multi-Object Dataset for Wide Baseline Multi-View Semantic Segmentation

MVMO: A Multi-Object Dataset for Wide Baseline Multi-View Semantic Segmentation

Automatic Image Generation Pipeline for Instance Segmentation of Deformable Linear Objects

MINERVAS: Massive INterior EnviRonments VirtuAl Synthesis

Contact Info

Product

Resources

About