Antipodal Robotic Grasping using Generative Residual Convolutional Neural Network

Kumra, Sulabh; Joshi, Shirin; Sahin, Ferat

doi:10.1109/iros45743.2020.9340777

Cited by 270 publications

(208 citation statements)

References 31 publications

Supporting

Mentioning

207

Contrasting

Order By: Relevance

“…Comparing to existing works, our proposal in this paper has wider applicability and higher integrity, consisting of a low-cost and reproducible resistance-based sensor, a general tactile-visual dataset, and a learning-based model. Our proposed dataset is also compatible with public datasets, which can be applied in existing learning models [6,7].…”

Section: Related Workmentioning

confidence: 99%

“…Grasp space representation: Conventional methods [6,7] define the grasping representation including the pose of object p = (x, y, z, x , y , z ), gripper's orientation angle , and opening width in Cartesian space (world/robot coordinates). For the planar grasping problem, we usually let the camera keep vertical to the tabletop, so the attitude of grasping ( x , y , z ) is fixed by (−90 • , 0, 0) in our robot system.…”

Section: Grasp Definitionmentioning

confidence: 99%

“…Conventional vision-based methods [6,7] formulate robotic grasp as a mapping modeling problem from perceptive space to grasp space: However, these methods do not consider the force for grasping, which could lead to grasping failure if the force is too small, or damage the object if the force is too large. The force required for grasping an object f is described in Eq.…”

Section: Problem Formulationmentioning

confidence: 99%

“…We augment the dataset like most supervised methods by rotating and scaling the raw data. We scale the RGB and depth image values, and gripper's opening width values in [0, 1] by normalization consistent with [6,7]. Finally, RGB and depth images are resized into 336 × 336 and fed into our network.…”

Section: Data Preprocessingmentioning

confidence: 99%

“…For example, [4,5] propose typical grasp detection datasets, which are widely used in vision-based robotic grasping tasks. Some other works [6,7] adopt a vision-based dataset to predict grasp points as a regression problem. But for the limitation of vision-based methods on force-sensitive tasks [3], tactile perception becomes an emerging modality for robotic grasp detection as a supplement to vision-based methods, however, previous studies have not given a general tactile-force dataset for this task.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Tactile–Visual Fusion Based Robotic Grasp Detection Method with a Reproducible Sensor

Song¹,

Luo²,

Yu³

2021

IJCIS

View full text Add to dashboard Cite

Robotic grasp detection is a fundamental problem in robotic manipulation. The conventional grasp methods, using vision information only, can cause potential damage in force-sensitive tasks. In this paper, we propose a tactile-visual based method using a reproducible sensor to realize a fine-grained and haptic grasping. Although there exist several tactile-based methods, they require expensive custom sensors in coordination with their specific datasets. In order to overcome the limitations, we introduce a low-cost and reproducible tactile fingertip and build a general tactile-visual fusion grasp dataset including 5,110 grasping trials. We further propose a hierarchical encoder-decoder neural network to predict grasp points and force in an end-to-end manner. Then comparisons of our method with the state-of-the-art methods in the benchmark are shown both in vision-based and tactile-visual fusion schemes, and our method outperforms in most scenarios. Furthermore, we also compare our fusion method with the only vision-based method in the physical experiment, and the results indicate that our end-to-end method empowers the robot with a more fine-grained grasp ability, reducing force redundancy by 41%. Our project is available at https://sites.google.com/view/tvgd

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Grasp Definitionmentioning

confidence: 99%

Section: Problem Formulationmentioning

confidence: 99%

Section: Data Preprocessingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Tactile–Visual Fusion Based Robotic Grasp Detection Method with a Reproducible Sensor

Song¹,

Luo²,

Yu³

2021

IJCIS

View full text Add to dashboard Cite

show abstract

Jensen–Shannon Divergence You Only Look Once: A Real‐Time Robotic Grasp Detection Network

Han,

2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

In this article, the arbitrary‐oriented object detection problem with application in robotic grasping is addressed. A novel Jensen–Shannon divergence (JSD)– You Only Look Once (YOLO) model is proposed, which enables real‐time grasp detection with high performance. The one‐stage object detection network YOLOv5 is modified with a decoupled head, which solves the angle classification problem and rectangle parameter regression problem separately, such that the YOLOv5 network is applicable for robotic grasping and the detection accuracy is significantly improved. A circular smooth label angle classification method is proposed to tackle the boundary discontinuity problem in angle regression, and the periodicity of the angle prediction is guaranteed. A novel Jensen–Shannon intersection of union is designed to calculate the intersection over union of oriented rectangles, which aims to better measure the discrepancies between the prediction and the ground truth and to avoid the singularity problem when two rectangles are not overlapped. Extensive evaluation on the Cornell and visual manipulation relationship dataset datasets demonstrates the effectiveness of the JSD–YOLO model in general robotic grasp operations, with 99.7% and 95.7% image‐wise split accuracy, respectively.

show abstract

Robot Grasping Detection Method Based on Keypoints

Yan,

Zhang

2024

Journal of Field Robotics

View full text Add to dashboard Cite

This study introduces a novel keypoint‐based grasp detection network, denoted as GKSCConv‐Net, which operates on n‐channel input images. The network architecture comprises three SCConv2D layers and three SCConvT2D layers. The SCConvT2D layers facilitate upsampling to maintain consistent dimensions between the output and input images. The resultant output consists of maps indicating left grasp points, right grasp points, and grasp center keypoints. The accuracy of predictions is enhanced through the incorporation of the keypoint refinement module and feature fusion module. To validate the model's generalization and applicability, comprehensive training, testing, and evaluation are conducted on diverse data sets, including the Cornell data set, Jacquard data set, and others representing real‐world scenarios. Furthermore, ablation experiments are employed to substantiate the efficacy of the spatial reconstruction unit (SRU) and channel reconstruction unit (CRU) within the SCConv, exploring their impact on grasp keypoint detection outcomes. Real robotic grasping experiments ultimately affirm the model's outstanding performance in practical settings.

show abstract

Antipodal Robotic Grasping using Generative Residual Convolutional Neural Network

Cited by 270 publications

References 31 publications

Tactile–Visual Fusion Based Robotic Grasp Detection Method with a Reproducible Sensor

Tactile–Visual Fusion Based Robotic Grasp Detection Method with a Reproducible Sensor

Jensen–Shannon Divergence You Only Look Once: A Real‐Time Robotic Grasp Detection Network

Robot Grasping Detection Method Based on Keypoints

Contact Info

Product

Resources

About