Graph-Based Visual Manipulation Relationship Reasoning Network for Robotic Grasping

Zuo, Guoyu; Tong, Jiayuan; Liu, Hongxing; Chen, Wenbai; Li, Jianfeng

doi:10.3389/fnbot.2021.719731

Cited by 16 publications

(6 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Simply inferring grasps without considering the underlying object arrangement can result in unsuccessful grasp attempts or even damaged objects. Recent work addresses this problem by attempting to learn the manipulation order for a picking system [27], [41]. However, currently only a few datasets are available that provide the necessary scene layout information [6], [13] (cf.…”

Section: Object Detection and Relationship Reasoningmentioning

confidence: 99%

MetaGraspNetV2: All-in-One Dataset Enabling Fast and Reliable Robotic Bin Picking via Object Relationship Reasoning and Dexterous Grasping

Gilles,

Chen,

Zeng

et al. 2024

IEEE Trans. Automat. Sci. Eng.

View full text Add to dashboard Cite

Grasping unknown objects in unstructured environments is one of the most challenging and demanding tasks for robotic bin picking systems. Developing a holistic approach is crucial to building such dexterous bin picking systems to meet practical requirements on speed, cost and reliability. Proposed datasets so far focus only on challenging sub-problems and are therefore limited in their ability to leverage the complementary relationship between individual tasks. In this paper, we tackle this holistic data challenge and design MetaGraspNetV2, an allin-one bin picking dataset consisting of (i) a photo-realistic dataset with over 296k images, which has been created through physics-based metaverse synthesis; and (ii) a real-world test dataset with 3.2k images featuring task-specific difficulty levels. Both datasets provide full annotations for amodal panoptic segmentation, object relationship detection, occlusion reasoning, 6-DoF pose estimation, and grasp detection for a parallel-jaw as well as a vacuum gripper. Extensive experiments demonstrate that our dataset outperforms state-of-the-art datasets in object detection, instance segmentation, amodal detection, parallel-jaw grasping, and vacuum grasping. Furthermore, leveraging the potential of our data for building holistic perception systems, we propose a single-shot-multi-pick (SSMP) grasping policy for scene understanding accelerated fast picking in high clutter. SSMP reasons about suitable manipulation orders for blindly picking multiple items given a single image acquisition. Physical robot experiments demonstrate that SSMP effectively speeds up cycle times through reducing image acquisitions by more than 47% while providing better grasp performance compared to state-of-the-art bin picking methods.

show abstract

Section: Object Detection and Relationship Reasoningmentioning

confidence: 99%

MetaGraspNetV2: All-in-One Dataset Enabling Fast and Reliable Robotic Bin Picking via Object Relationship Reasoning and Dexterous Grasping

Gilles,

Chen,

Zeng

et al. 2024

IEEE Trans. Automat. Sci. Eng.

View full text Add to dashboard Cite

show abstract

“…Recently, graph neural networks (GNNs) are widely proved to be efficient for detecting relationships among objects [251,277]. [284] proposed a GNN-based method for VMR detection and achieved better performance. Such kind of methods has been successfully applied to the decision of grasping sequence in dense-clutter scenes [182,267,272].…”

Section: Relational Grasp Synthesismentioning

confidence: 99%

Robotic Grasping from Classical to Modern: A Survey

Zhang¹,

Tang²,

Sun³

et al. 2022

Preprint

View full text Add to dashboard Cite

Robotic Grasping has always been an active topic in robotics since grasping is one of the fundamental but most challenging skills of robots. It demands the coordination of robotic perception, planning, and control for robustness and intelligence. However, current solutions are still far behind humans, especially when confronting unstructured scenarios.In this paper, we survey the advances of robotic grasping, starting from the classical formulations and solutions to the modern ones. By reviewing the history of robotic grasping, we want to provide a complete view of this community, and perhaps inspire the combination and fusion of different ideas, which we think would be helpful to touch and explore the essence of robotic grasping problems. In detail, we firstly give an overview of the analytic methods for robotic grasping. After that, we provide a discussion on the recent state-ofthe-art data-driven grasping approaches rising in recent years. With the development of computer vision, semantic grasping is being widely investigated and can be the basis of intelligent manipulation and skill learning for autonomous robotic systems in the future.Therefore, in our survey, we also briefly review the recent progress in this topic. Finally, we discuss the open problems and the future research directions that may be important for the human-level robustness, autonomy, and intelligence of robots.

show abstract

“…A construction scene graph contains not only intuitive visual information, but also deep semantic information. Scene graphs are one of the methods used to construct the visual relations of images [23]. The main idea is to divide the visual relations between all objects in an image into a triadic subject-predicate-object form, which is used as a whole learning task [24,25].…”

Section: Semantic Conversion Module Based On Frequency Baselinementioning

confidence: 99%

Visual Relationship-Based Identification of Key Construction Scenes on Highway Bridges

Wang

Geng

et al. 2022

Buildings

View full text Add to dashboard Cite

Highway bridges play an important role in traffic construction; however, accidents caused by bridge construction occur frequently, resulting in significant loss of life and property. The identification of bridge construction scenes not only keeps track of the construction progress, but also enables real-time monitoring of the construction process and the timely detection of safety hazards. This paper proposes a deep learning method in artificial intelligence (AI) for identifying key construction scenes of highway bridges based on visual relationships. First, based on the analysis of bridge construction characteristics and construction process, five key construction scenes are selected. Then, by studying the underlying features of the five scenes, a construction scene identification feature information table is built, and construction scene identification rules are formulated. Afterward, a bridge key construction scene identification model (CSIN) is built; this model comprises target detection, visual relationship extraction, semantic conversion, scene information fusion, and identification results output. Finally, the effectiveness of the proposed method is verified experimentally. The results show that the proposed method can effectively identify key construction scenes for highway bridges with an accuracy rate of 94%, and enable the remote intelligent monitoring of highway bridge construction processes to ensure that projects are carried out safely.

show abstract

Graph-Based Visual Manipulation Relationship Reasoning Network for Robotic Grasping

Cited by 16 publications

References 19 publications

MetaGraspNetV2: All-in-One Dataset Enabling Fast and Reliable Robotic Bin Picking via Object Relationship Reasoning and Dexterous Grasping

MetaGraspNetV2: All-in-One Dataset Enabling Fast and Reliable Robotic Bin Picking via Object Relationship Reasoning and Dexterous Grasping

Robotic Grasping from Classical to Modern: A Survey

Visual Relationship-Based Identification of Key Construction Scenes on Highway Bridges

Contact Info

Product

Resources

About