MetaGraspNet: A Large-Scale Benchmark Dataset for Scene-Aware Ambidextrous Bin Picking via Physics-based Metaverse Synthesis

Gilles, Maximilian; Chen, Yuhao; Winter, Tim Robin; Zeng, E. Zhixuan; Wong, Alexander

doi:10.1109/case49997.2022.9926427

Cited by 11 publications

(6 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our previous work [17], the potential of our synthetic data for vacuum grasping and class-agnostic instance segmentation has been demonstrated. In addition, the proposed vacuum seal model was able to generalize to different cup materials and dimensions with an average seal precision of 95.0% and outperformed state-of-the-art models, e.g.…”

Section: Methodsmentioning

confidence: 99%

“…Collecting such comprehensive label sets from experiments [15], [16] or manually [11] is too expensive and time prohibitive. Motivated by work demonstrating the generalization capabilities towards real-world data [3], [13], [14], our previous work [17] proposed a data creation pipeline based on metaverse synthesis and contributed two datasets: a large-scale synthetic training dataset and smaller, but comprehensive real-world test dataset.…”

Section: Fast Cycle Timesmentioning

confidence: 99%

“…Our proposed method for synthetic data generation, initially introduced in [17], can be divided into three steps (cf. Fig.…”

Section: Synthetic Datamentioning

confidence: 99%

“…It is concluded that there is a leak between object and vacuum cup if one of the mass points is not in contact with the object's surface. A more detailed derivation of our model and experimental evaluation can be found in our previous work [17].…”

Section: B Grasps Sampling Strategymentioning

confidence: 99%

“…In our previous work [17], the effectiveness of our data generation pipeline has been evaluated by comparing the grasp performance of the vacuum grasp network SuctionNet-1Billion [2] trained on their proposed large-scale real-world dataset versus a version of SuctionNet-1Billion trained on our synthetic data. The results showed clearly that our synthetic data improves picking performance for unseen objects.…”

Section: A Vacuum Grasp Experimentsmentioning

confidence: 99%

See 4 more Smart Citations

MetaGraspNetV2: All-in-One Dataset Enabling Fast and Reliable Robotic Bin Picking via Object Relationship Reasoning and Dexterous Grasping

Gilles,

Chen,

Zeng

et al. 2024

IEEE Trans. Automat. Sci. Eng.

View full text Add to dashboard Cite

Grasping unknown objects in unstructured environments is one of the most challenging and demanding tasks for robotic bin picking systems. Developing a holistic approach is crucial to building such dexterous bin picking systems to meet practical requirements on speed, cost and reliability. Proposed datasets so far focus only on challenging sub-problems and are therefore limited in their ability to leverage the complementary relationship between individual tasks. In this paper, we tackle this holistic data challenge and design MetaGraspNetV2, an allin-one bin picking dataset consisting of (i) a photo-realistic dataset with over 296k images, which has been created through physics-based metaverse synthesis; and (ii) a real-world test dataset with 3.2k images featuring task-specific difficulty levels. Both datasets provide full annotations for amodal panoptic segmentation, object relationship detection, occlusion reasoning, 6-DoF pose estimation, and grasp detection for a parallel-jaw as well as a vacuum gripper. Extensive experiments demonstrate that our dataset outperforms state-of-the-art datasets in object detection, instance segmentation, amodal detection, parallel-jaw grasping, and vacuum grasping. Furthermore, leveraging the potential of our data for building holistic perception systems, we propose a single-shot-multi-pick (SSMP) grasping policy for scene understanding accelerated fast picking in high clutter. SSMP reasons about suitable manipulation orders for blindly picking multiple items given a single image acquisition. Physical robot experiments demonstrate that SSMP effectively speeds up cycle times through reducing image acquisitions by more than 47% while providing better grasp performance compared to state-of-the-art bin picking methods.

show abstract

Section: Methodsmentioning

confidence: 99%