Deep Learning Approaches to Grasp Synthesis: A Review

Newbury, Rhys; Gu, Morris; Lachlan, Chumbley,; Mousavian, Arsalan; Eppner, Clemens; Leitner, Jürgen; Bohg, Jeannette; Morales, Antonio; Asfour, Tamim; Kragić, Danica; Fox, Dieter; Cosgun, Akansel

doi:10.1109/tro.2023.3280597

Cited by 79 publications

(9 citation statements)

References 194 publications

(451 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The measure of graspability is calculated by convolving the mask image with a depth map that has been converted into a binary form. The threshold for each region differs based on the minimum height of 3D points in the region and the length of the gripper [ 16 , 17 , 18 ]. The proposed approach is appropriate for general objects since it does not presume any 3D model of the object [ 19 , 20 ].…”

Section: Related Workmentioning

confidence: 99%

Object Recognition and Grasping for Collaborative Robots Based on Vision

Sun,

Wu,

Zhao

et al. 2023

Sensors

View full text Add to dashboard Cite

This study introduces a parallel YOLO–GG deep learning network for collaborative robot target recognition and grasping to enhance the efficiency and precision of visual classification and grasping for collaborative robots. First, the paper outlines the target classification and detection task, the grasping system of the robotic arm, and the dataset preprocessing method. The real-time recognition and grasping network can identify a diverse spectrum of unidentified objects and determine the target type and appropriate capture box. Secondly, we propose a parallel YOLO–GG deep vision network based on YOLO and GG-CNN. Thirdly, the YOLOv3 network, pre-trained with the COCO dataset, identifies the object category and position, while the GG-CNN network, trained using the Cornell Grasping dataset, predicts the grasping pose and scale. This study presents the processes for generating a target’s grasping frame and recognition type using GG-CNN and YOLO networks, respectively. This completes the investigation of parallel networks for target recognition and grasping in collaborative robots. Finally, the experimental results are evaluated on the self-constructed NEU-COCO dataset for target recognition and positional grasping. The speed of detection has improved by 14.1%, with an accuracy of 94%. This accuracy is 4.0% greater than that of YOLOv3. Experimental proof was obtained through a robot grasping actual objects.

show abstract

Section: Related Workmentioning

confidence: 99%

Object Recognition and Grasping for Collaborative Robots Based on Vision

Sun,

Wu,

Zhao

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…In this section, we briefly summarize the major achievements of planar and 6-DoF grasping datasets. A more thorough list can be found in a recent review (Newbury et al 2022).…”

Section: Related Workmentioning

confidence: 99%

Robust grasping across diverse sensor qualities: The GraspNet-1Billion dataset

Fang,

Gou,

Wang

et al. 2023

The International Journal of Robotics Research

View full text Add to dashboard Cite

Robust object grasping in cluttered scenes is vital to all robotic prehensile manipulation. In this paper, we present the GraspNet-1Billion benchmark that contains rich real-world captured cluttered scenarios and abundant annotations. This benchmark aims at solving two critical problems for the cluttered scenes parallel-finger grasping: the insufficient real-world training data and the lacking of evaluation benchmark. We first contribute a large-scale grasp pose detection dataset. Two different depth cameras based on structured-light and time-of-flight technologies are adopted. Our dataset contains 97,280 RGB-D images with over one billion grasp poses. In total, 190 cluttered scenes are collected, among which 100 are training set and 90 are for testing. Meanwhile, we build an evaluation system that is general and user-friendly. It directly reports a predicted grasp pose’s quality by analytic computation, which is able to evaluate any kind of grasp representation without exhaustively labeling the ground-truth. We further divide the test set into three difficulties to better evaluate algorithms’ generalization ability. Our dataset, accessing API and evaluation code, are publicly available at www.graspnet.net.

show abstract

“…When a robot interacts with the real world, grasping and manipulating objects is an integral part of this task. Reviewing previous robot grasping research, the focus of robot grasping has gradually shifted from multi-fingered contact-based representations to pose-based ones [1]. With the widespread attention paid to the application of computer vision in robotics and the important role played by point clouds in several fields * Author to whom any correspondence should be addressed.…”

Section: Introductionmentioning

confidence: 99%

FastGNet: an efficient 6-DOF grasp detection method with multi-attention mechanisms and point transformer network

Ding,

Wang,

Gao

et al. 2024

Meas. Sci. Technol.

View full text Add to dashboard Cite

A pivotal technology for autonomous robot grasping is efficient and accurate grasp pose detection, which enables robotic arms to grasp objects in cluttered environments without human intervention. However, most existing methods rely on PointNet or CNN as backbones for grasp pose prediction, which may lead to unnecessary computational overhead on invalid grasp points or background information. Consequently, performing efficient grasp pose detection for graspable points in complex scenes becomes a challenge. In this paper, we propose FastGNet, an end-to-end model that combines multiple attention mechanisms with the Transformer architecture to generate 6-DOF grasp poses efficiently. Our approach involves a novel sparse point cloud voxelization technique, preserving the complete mapping between points and voxels while generating positional embeddings for the Transformer network. By integrating unsupervised and supervised attention mechanisms into the grasp model, our method significantly improves the performance of focusing on graspable target points in complex scenes. The effectiveness of FastGNet is validated on the large-scale GraspNet-1Billion dataset. Our approach outperforms previous methods and achieves relatively fast inference times, highlighting its potential to advance autonomous robot grasping capabilities.

show abstract

Deep Learning Approaches to Grasp Synthesis: A Review

Cited by 79 publications

References 194 publications

Object Recognition and Grasping for Collaborative Robots Based on Vision

Object Recognition and Grasping for Collaborative Robots Based on Vision

Robust grasping across diverse sensor qualities: The GraspNet-1Billion dataset

FastGNet: an efficient 6-DOF grasp detection method with multi-attention mechanisms and point transformer network

Contact Info

Product

Resources

About