GraspVDN: scene-oriented grasp estimation by learning vector representations of grasps

Dong, Zhipeng; Tian, Hongkun; Bao, Xuefeng; Yan, Yunhui; Chen, Fei

doi:10.1007/s40747-021-00459-x

Cited by 9 publications

(2 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…point cloud [2] and RGB image [3]) to rearrange a deformable object into a prescribed goal configuration. Different from rigid manipulation [4][5][6], deformable rearrangement poses two new challenges. The first challenge lies in the high dimensionality of the deformable configuration space [7].…”

Section: Introductionmentioning

confidence: 99%

Learning visual-based deformable object rearrangement with local graph neural networks

Deng

Wang

Chen

2023

Complex Intell. Syst.

View full text Add to dashboard Cite

Goal-conditioned rearrangement of deformable objects (e.g. straightening a rope and folding a cloth) is one of the most common deformable manipulation tasks, where the robot needs to rearrange a deformable object into a prescribed goal configuration with only visual observations. These tasks are typically confronted with two main challenges: the high dimensionality of deformable configuration space and the underlying complexity, nonlinearity and uncertainty inherent in deformable dynamics. To address these challenges, we propose a novel representation strategy that can efficiently model the deformable object states with a set of keypoints and their interactions. We further propose local-graph neural network (GNN), a light local GNN learning to jointly model the deformable rearrangement dynamics and infer the optimal manipulation actions (e.g. pick and place) by constructing and updating two dynamic graphs. Both simulated and real experiments have been conducted to demonstrate that the proposed dynamic graph representation shows superior expressiveness in modeling deformable rearrangement dynamics. Our method reaches much higher success rates on a variety of deformable rearrangement tasks (96.3% on average) than state-of-the-art method in simulation experiments. Besides, our method is much more lighter and has a 60% shorter inference time than state-of-the-art methods. We also demonstrate that our method performs well in the multi-task learning scenario and can be transferred to real-world applications with an average success rate of 95% by solely fine tuning a keypoint detector. A supplementary video can be found at https://youtu.be/AhwTQo6fCM0.

show abstract

Section: Introductionmentioning

confidence: 99%

Learning visual-based deformable object rearrangement with local graph neural networks

Deng

Wang

Chen

2023

Complex Intell. Syst.

View full text Add to dashboard Cite

show abstract

“…Moreover, embedded with an online evaluation system, our benchmark is able to evaluate current mainstream grasp detection algorithms in a unified manner. So far, the dataset has been adopted for planar (Dong et al 2022;Kumra et al 2022) and 6-DoF (Gou et al 2021;Wang et al 2021) grasp pose detection, point cloud domain adaptation (Shen et al 2022), haptic based object recognition (Sintov and Ben-David 2022), etc.…”

Section: Introductionmentioning

confidence: 99%

Robust grasping across diverse sensor qualities: The GraspNet-1Billion dataset

Fang,

Gou,

Wang

et al. 2023

The International Journal of Robotics Research

View full text Add to dashboard Cite

Robust object grasping in cluttered scenes is vital to all robotic prehensile manipulation. In this paper, we present the GraspNet-1Billion benchmark that contains rich real-world captured cluttered scenarios and abundant annotations. This benchmark aims at solving two critical problems for the cluttered scenes parallel-finger grasping: the insufficient real-world training data and the lacking of evaluation benchmark. We first contribute a large-scale grasp pose detection dataset. Two different depth cameras based on structured-light and time-of-flight technologies are adopted. Our dataset contains 97,280 RGB-D images with over one billion grasp poses. In total, 190 cluttered scenes are collected, among which 100 are training set and 90 are for testing. Meanwhile, we build an evaluation system that is general and user-friendly. It directly reports a predicted grasp pose’s quality by analytic computation, which is able to evaluate any kind of grasp representation without exhaustively labeling the ground-truth. We further divide the test set into three difficulties to better evaluate algorithms’ generalization ability. Our dataset, accessing API and evaluation code, are publicly available at www.graspnet.net.

show abstract

Estimation of Optimal Gripper Configuration Through an Embedded Array of Proximity Sensors

Pereira,

Joventino,

Fabro

et al. 2024

Communications in Computer and Information Science

View full text Add to dashboard Cite

GraspVDN: scene-oriented grasp estimation by learning vector representations of grasps

Cited by 9 publications

References 39 publications

Learning visual-based deformable object rearrangement with local graph neural networks

Learning visual-based deformable object rearrangement with local graph neural networks

Robust grasping across diverse sensor qualities: The GraspNet-1Billion dataset

Estimation of Optimal Gripper Configuration Through an Embedded Array of Proximity Sensors

Contact Info

Product

Resources

About