Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items

Downs, Laura; Francis, Anthony G.; Koenig, Nate; Brandon, Kinman,; Hickman, Ryan; Krista, Reymann,; McHugh, Thomas Barlow; Vanhoucke, Vincent

doi:10.1109/icra46639.2022.9811809

Cited by 160 publications

(40 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The dataset consists of 236 training and 22 test sequences, each containing the images captured with the polarizer, together with the events generated from these, as well as the groundtruth surface normal provided by the renderer. Meshes used from data generation were obtained from the Google Scanned Objects dataset [9] and the textures derived from [6]. More details are provided in the supplementary material.…”

Section: Esfp-synthetic Datasetmentioning

confidence: 99%

See 1 more Smart Citation

Event-based Shape from Polarization

Muglikar¹,

Bauersfeld²,

Moeys³

et al. 2023

Preprint

View full text Add to dashboard Cite

State-of-the-art solutions for Shape-from-Polarization (SfP) suffer from a speed-resolution tradeoff: they either sacrifice the number of polarization angles measured or necessitate lengthy acquisition times due to framerate constraints, thus compromising either accuracy or latency. We tackle this tradeoff using event cameras. Event cameras operate at microseconds resolution with negligible motion blur, and output a continuous stream of events that precisely measures how light changes over time asynchronously. We propose a setup that consists of a linear polarizer rotating at high-speeds in front of an event camera. Our method uses the continuous event stream caused by the rotation to reconstruct relative intensities at multiple polarizer angles. Experiments demonstrate that our method outperforms physics-based baselines using frames, reducing the MAE by 25% in synthetic and real-world dataset. In the real world, we observe, however, that the challenging conditions (i.e., when few events are generated) harm the performance of physics-based solutions. To overcome this, we propose a learning-based approach that learns to estimate surface normals even at low event-rates, improving the physicsbased approach by 52% on the real world dataset. The proposed system achieves an acquisition speed equivalent to 50 fps (>twice the framerate of the commercial polarization sensor) while retaining the spatial resolution of 1MP. Our evaluation is based on the first large-scale dataset for event-based SfP.

show abstract

Section: Esfp-synthetic Datasetmentioning

confidence: 99%

“…In this section, we provide details on the ESfP-Synthetic dataset which we use for evaluation. The dataset was generated using publicly available meshes [9] which consists of over 1000 3D scanned common household objects. These meshes were textured using the 25 textures available in this dataset [6].…”

Section: Esfp-synthetic Datasetmentioning

confidence: 99%

Event-based Shape from Polarization

Muglikar¹,

Bauersfeld²,

Moeys³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Dynamic objects. There are several datasets about objects [30], being those scanned or actual CAD models/meshes. Among those, Google Scanned Objects [30], with its realistically looking household objects, and ShapeNet [31], with its variety, are two good and popular examples of datasets that represent these two categories.…”

Section: B Dynamic Contentmentioning

confidence: 99%

GRADE: Generating Realistic Animated Dynamic Environments for Robotics Research

Bonetto¹,

Xu²,

Ahmad³

2023

Preprint

View full text Add to dashboard Cite

Simulation engines like Gazebo, Unity and Webots are widely adopted in robotics. However, they lack either full simulation control, ROS integration, realistic physics, or photorealism. Recently, synthetic data generation and realistic rendering advanced tasks like target tracking and human pose estimation. However, when focusing on vision applications, there is usually a lack of information like sensor measurements (e.g. IMU, LiDAR, joint state), or time continuity. On the other hand, simulations for most robotics applications are obtained in (semi)static environments, with specific sensor settings and low visual fidelity. In this work, we present a solution to these issues with a fully customizable framework for generating realistic animated dynamic environments (GRADE) for robotics research. The data produced can be post-processed, e.g. to add noise, and easily expanded with new information using the tools that we provide. To demonstrate GRADE, we use it to generate an indoor dynamic environment dataset and then compare different SLAM algorithms on the produced sequences. By doing that, we show how current research over-relies on well-known benchmarks and fails to generalize. Furthermore, our tests with YOLO and Mask R-CNN provide evidence that our data can improve training performance and generalize to real sequences. Finally, we show GRADE's flexibility by using it for indoor active SLAM, with diverse environment sources, and in a multi-robot scenario. In doing that, we employ different control, asset placement, and simulation techniques. The code, results, implementation details, and generated data are provided as open-source. The main project page is https://eliabntt.github.io/grade-rr while the accompanying video can be found at https://youtu.be/cmywCSD-9TU.

show abstract

“…In 3D vision, datasets like ModelNet [74] and ShapeNet [8] focus on modeling the geometry of objects but without realistic visual textures. Recently, with the popularity of neural rendering approaches [50,62], a series of 3D datasets are introduced with both realistic shape and appearance, such as CO3D [58], Google Scanned Objects [21], and ABO [15]. Unlike all datasets above that focus only on the visual modality, we also model the acoustic and tactile modalities of objects.…”

Section: Related Workmentioning

confidence: 99%

ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer

Gao¹,

Chang³

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

We introduce the OBJECTFOLDER BENCHMARK, a benchmark suite of 10 tasks for multisensory object-centric learning, centered around object recognition, reconstruction, and manipulation with sight, sound, and touch. We also introduce the OBJECTFOLDER REAL dataset, including the multisensory measurements for 100 real-world household objects, building upon a newly designed pipeline for collecting the 3D meshes, videos, impact sounds, and tactile readings of real-world objects. We conduct systematic benchmarking on both the 1,000 multisensory neural objects from OBJECTFOLDER, and the real multisensory data from OBJECTFOLDER REAL. Our results demonstrate the importance of multisensory perception and reveal the respective roles of vision, audio, and touch for different object-centric learning tasks. By publicly releasing our dataset and benchmark suite, we hope to catalyze and enable new research in multisensory object-centric learning in computer vision, robotics, and beyond. Project page: https://objectfolder.stanford.edu * indicates equal contribution. † Yiming is affiliated with Shanghai Jiao Tong University. The work was done when he was visiting Stanford University as a summer intern.

show abstract

Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items

Cited by 160 publications

References 28 publications

Event-based Shape from Polarization

Event-based Shape from Polarization

GRADE: Generating Realistic Animated Dynamic Environments for Robotics Research

ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer

Contact Info

Product

Resources

About