ABO: Dataset and Benchmarks for Real-World 3D Object Understanding

Collins, Jasmine D.; Goel, Shubham; Deng, Kenan; Luthra, Achleshwar; Xu, Leon; Gundogdu, Erhan; Zhang, Xi; Vicente, Tomás F. Yago; Dideriksen, T.L.; Arora, Himanshu; Guillaumin, Matthieu; Malik, Jitendra

doi:10.48550/arxiv.2110.06199

Cited by 8 publications

(11 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…BigBIRD [62] and YCB [10] directly model real-world objects but only for a small number of object instances. ABO [14] was recently introduced, containing 3D models for over 8K objects of real household objects, but it focuses only on the visual modality, similar to the other datasets above.…”

Section: Related Workmentioning

confidence: 99%

“…To this end, we introduce OBJECTFOLDER 2.0, a large dataset of implicitly represented multisensory replicas of real-world objects. It contains 1,000 high-quality 3D objects collected from online repositories [1,2,10,14]. Compared with OBJECTFOLDER 1.0 1 that is slow in rendering and of limited quality in multisensory simulation, we improve the acoustic and tactile simulation pipelines to render more realistic multisensory data.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer

Gao¹,

Si²,

Chang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Objects play a crucial role in our everyday activities. Though multisensory object-centric learning has shown great potential lately, the modeling of objects in prior work is rather unrealistic. OBJECTFOLDER 1.0 is a recent dataset that introduces 100 virtualized objects with visual, acoustic, and tactile sensory data. However, the dataset is small in scale and the multisensory data is of limited quality, hampering generalization to real-world scenarios. We present OBJECTFOLDER 2.0, a large-scale, multisensory dataset of common household objects in the form of implicit neural representations that significantly enhances OBJECTFOLDER 1.0 in three aspects. First, our dataset is 10 times larger in the amount of objects and orders of magnitude faster in rendering time. Second, we significantly improve the multisensory rendering quality for all three modalities. Third, we show that models learned from virtual objects in our dataset successfully transfer to their real-world counterparts in three challenging tasks: object scale estimation, contact localization, and shape reconstruction. OBJECTFOLDER 2.0 offers a new path and testbed for multisensory learning in computer vision and robotics. The dataset is available at https://github. com/rhgao/ObjectFolder.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer

Gao¹,

Si²,

Chang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…To evaluate our method on complex shapes, we generate camouflage textures for a dataset of 49 animal meshes from [59]. We also provide a qualitative furniture shape from [11] (Fig. 1).…”

Section: Datasetmentioning

confidence: 99%

GANmouflage: 3D Object Nondetection with Texture Fields

Guo¹,

Collins²,

Lima³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

https://rrrrrguo.github.io/ganmouflage Figure 1. Learning multi-view camouflage. Our model learns to hide 3D objects within scenes. Given an object's shape, position, and a distribution of possible viewpoints it will be seen from, we estimate a texture field that camouflages it. We show example outputs from our model above, with two viewpoints for each camouflaged object. Can you find them all? See Figure 2 for answers. We chose views in which the object was relatively conspicuous and cropped the images to make objects easier to see. Please see the videos on our webpage for more examples.

show abstract

“…The tabletop objects in TO-Scene are originated from ModelNet [51] containing 151,128 3D CAD models of 660 categories, and ShapeNetCore [5] covering 55 object classes with 51,300 3D models. As for 3D shape datasets, [5,51,25,3,52,53] provide CAD models, while [7,43,34,48,9] advocate the realistic data. [26,39] contain multi-view images of 3D objects, and the recent Objectron [1] is a collection of short object-centric videos.…”

Section: Related Workmentioning

confidence: 99%

TO-Scene: A Large-scale Dataset for Understanding 3D Tabletop Scenes

Xu¹,

Chen²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

TO_ScanNet o b j in w h o le r o o m Transfer obj CADs onto real tables, by crowdsourcing Web UI Object CADs from ModelNet or ShapeNet TO_Vanilla Real Tables from ScanNet Source Datasets TO-Real Test up load sim to real autolabel TO_Crowd in cr ea se ob j TO-Scene BuildFigure 1: Overview of data acquisition framework and TO-Scene dataset. We firstly transfer object CADs [51, 5] onto real tables [11] via crowdsourcing Web UI. Then the tabletop scenes are simluated into real scans and annotated automatically. Three variants (TO_Vanilla, TO_Crowd and TO_ScanNet) are presented for various scenarios. A real-scanned test data TO-Real is provided to verify the practical value of TO-Scene.

show abstract

ABO: Dataset and Benchmarks for Real-World 3D Object Understanding

Abstract: catalog images high-resolution geometry physically-based renderingsFigure 1. ABO is a dataset of product images and realistic, high-resolution, physically-based 3D models of household objects.

Cited by 8 publications

References 47 publications

ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer

ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer

GANmouflage: 3D Object Nondetection with Texture Fields

TO-Scene: A Large-scale Dataset for Understanding 3D Tabletop Scenes

Contact Info

Product

Resources

About