REGRAD: A Large-Scale Relational Grasp Dataset for Safe and Object-Specific Robotic Grasping in Clutter

Zhang, Hanbo; Yang, Deyu; Wang, Han; Zhao, Binglei; Lan, Xuguang; Ding, Jishiyu; Zheng, Nanning

doi:10.1109/lra.2022.3142401

Cited by 29 publications

(19 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To solve the problem of data, self-supervised learning and unsupervised learning are extensively explored in recent years [15,111], including some excellent works in robotic grasping [17,18,154,188,265]. It is also possible to train grasping policies in physical simulators and then transfer to the real world [103,105,257,258,273,278]. By enough data, the performance of data-driven approaches substantially outperforms the classical methods.…”

Section: Introductionmentioning

confidence: 99%

Robotic Grasping from Classical to Modern: A Survey

Zhang¹,

Tang²,

Sun³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Robotic Grasping has always been an active topic in robotics since grasping is one of the fundamental but most challenging skills of robots. It demands the coordination of robotic perception, planning, and control for robustness and intelligence. However, current solutions are still far behind humans, especially when confronting unstructured scenarios.In this paper, we survey the advances of robotic grasping, starting from the classical formulations and solutions to the modern ones. By reviewing the history of robotic grasping, we want to provide a complete view of this community, and perhaps inspire the combination and fusion of different ideas, which we think would be helpful to touch and explore the essence of robotic grasping problems. In detail, we firstly give an overview of the analytic methods for robotic grasping. After that, we provide a discussion on the recent state-ofthe-art data-driven grasping approaches rising in recent years. With the development of computer vision, semantic grasping is being widely investigated and can be the basis of intelligent manipulation and skill learning for autonomous robotic systems in the future.Therefore, in our survey, we also briefly review the recent progress in this topic. Finally, we discuss the open problems and the future research directions that may be important for the human-level robustness, autonomy, and intelligence of robots.

show abstract

Section: Introductionmentioning

confidence: 99%

Robotic Grasping from Classical to Modern: A Survey

Zhang¹,

Tang²,

Sun³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Finding reliable grasps is therefore not limited to reason about the physical interaction between gripper and object, but also challenges the system to understand the arrangement of objects in the scene and identify an appropriate picking sequence. While many researches have provided excellent datasets on each individual part such as SynPick [1] for pose estimation and gripper-object interaction, REGRAD [2] for relationship reasoning, SuctionNet-1Billion [3] for vacuum grasping or Dex-Net 4.0 [4] for ambidextrous grasping, the problems were only addressed from one side or did not target automation. Because robotic picking is a multi-stage complex problem, solving the problem from one aspect will miss the details of other aspects.…”

Section: Introductionmentioning

confidence: 99%

“…Collecting such comprehensive data from real robot experiments [5], [6] or manually [7] would be too expensive and time prohibitive and as such intractable from a scalability perspective. Motivated by earlier work in the field of synthetic data generation [1], [2] and inspired from the recent rise in the concept of metaverses, we introduce MetaGraspNet: a large-scale photo-realistic physics-based bin picking dataset for ambidextrous grasping and bin scene reasoning. By providing rich semantic scene labels such as amodal segmentation masks or object layout graphs together with heterogeneous grasp labels, object poses, and keypoint labels, MetaGraspNet challenges picking systems to take the next step towards multi-gripper usage and cognitive understanding of bin scenes.…”

Section: Introductionmentioning

confidence: 99%

“…Since generating such grasp labels in SE(3) can become very tedious, recent 6 DoF datasets rely on automatic sampling schemes for grasp candidates. The generation of these labels is either based on analytical models such as antipodal-based samplers used in [12], [2] or physics simulation which combines analytical sampling with physics simulators [4], [18]. For an in-depth overview we refer to [25].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

MetaGraspNet: A Large-Scale Benchmark Dataset for Scene-Aware Ambidextrous Bin Picking via Physics-based Metaverse Synthesis

Gilles¹,

Chen²,

Winter³

et al. 2022

Preprint

View full text Add to dashboard Cite

Autonomous bin picking poses significant challenges to vision-driven robotic systems given the complexity of the problem, ranging from various sensor modalities, to highly entangled object layouts, to diverse item properties and gripper types. Existing methods often address the problem from one perspective. Diverse items and complex bin scenes require diverse picking strategies together with advanced reasoning. As such, to build robust and effective machine-learning algorithms for solving this complex task requires significant amounts of comprehensive and high quality data. Collecting such data in real world would be too expensive and time prohibitive and therefore intractable from a scalability perspective. To tackle this big, diverse data problem, we take inspiration from the recent rise in the concept of metaverses, and introduce MetaGraspNet, a large-scale photo-realistic bin picking dataset constructed via physics-based metaverse synthesis. The proposed dataset contains 217k RGBD images across 82 different article types, with full annotations for object detection, amodal perception, keypoint detection, manipulation order and ambidextrous grasp labels for a parallel-jaw and vacuum gripper. We also provide a real dataset consisting of over 2.3k fully annotated high-quality RGBD images, divided into 5 levels of difficulties and an unseen object set to evaluate different object and layout properties. Finally, we conduct extensive experiments showing that our proposed vacuum seal model and synthetic dataset achieves state-of-the-art performance and generalizes to real world use-cases.

show abstract

“…With environment layout labels, pick planning can be trained more intelligently to avoid object damages [14,16,8]. However, there are only a few datasets [14,8,13] providing layout labels. Most of the datasets [14,8] have limited training value as they lack in data size, depth information, as well as segmentation labels.…”

Section: Introductionmentioning

confidence: 99%

MetaGraspNet: A Large-Scale Benchmark Dataset for Vision-driven Robotic Grasping via Physics-based Metaverse Synthesis

Chen¹,

Zeng²,

Gilles³

et al. 2021

Preprint

View full text Add to dashboard Cite

There has been increasing interest in smart factories powered by robotics systems to tackle repetitive, laborious tasks. One particular impactful yet challenging task in robotics-powered smart factory applications is robotic grasping: using robotic arms to grasp objects autonomously in different settings. Robotic grasping requires a variety of computer vision tasks such as object detection, segmentation, grasp prediction, pick planning, etc. While significant progress has been made in leveraging of machine learning for robotic grasping, particularly with deep learning, a big challenge remains in the need for large-scale, high-quality RGBD datasets that cover a wide diversity of scenarios and permutations.

show abstract

REGRAD: A Large-Scale Relational Grasp Dataset for Safe and Object-Specific Robotic Grasping in Clutter

Cited by 29 publications

References 44 publications

Robotic Grasping from Classical to Modern: A Survey

Robotic Grasping from Classical to Modern: A Survey

MetaGraspNet: A Large-Scale Benchmark Dataset for Scene-Aware Ambidextrous Bin Picking via Physics-based Metaverse Synthesis

MetaGraspNet: A Large-Scale Benchmark Dataset for Vision-driven Robotic Grasping via Physics-based Metaverse Synthesis

Contact Info

Product

Resources

About