SE-ResUNet: A Novel Robotic Grasp Detection Method

Yu, Sheng; Zhai, Di‐Hua; Xia, Yuanqing; Wu, Haoran; Liao, Jun

doi:10.1109/lra.2022.3145064

Cited by 72 publications

(30 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As shown in the figure, the quality heatmaps demonstrate the robustness of our proposed method, which contributes to the superior performance of our grasp detection results. We also conducted a comparative analysis of our grasp detection algorithm with that of several other methods [5,6,[8][9][10][11][12] using the Jacquard dataset. Table 3 presents the statistical results of our experiment with the Jacquard dataset.…”

Section: Cornell Dataset Experiments Resultsmentioning

confidence: 99%

“…Similarly, in 2022, H. Cao et al [3] proposed a Gaussian-based grasp representation method using a generative grasping detection model that incorporates both RGB and depth images as inputs. Also in 2022, S. Yu et al [9] introduced another approach using a residual neural network and squeeze-and-excitation modules.…”

Section: Multiple Modality Fusion Based Grasp Detectionmentioning

confidence: 99%

See 1 more Smart Citation

Bilateral Cross-Modal Fusion Network for Robot Grasp Detection

Zhang¹,

Sun²,

Liu³

et al. 2023

Preprint

View full text Add to dashboard Cite

In the field of vision-based robot grasping, effectively leveraging RGB and depth information to accurately determine the position and pose of a target is a critical issue. To address this challenge, we propose a tri-stream cross-modal fusion architecture for 2-DoF visual grasp detection. This architecture facilitates the interaction of RGB and depth bilateral information and is designed to efficiently aggregate multiscale information. Our novel modal interaction module (MIM) with spatial-wise cross-attention algorithm adaptively captures cross-modal feature information. Meanwhile, the channel interaction modules (CIM) further enhance the aggregation of different modal streams. In addition, we efficiently aggregate global multiscale information through a hierarchical structure with skipping connections. To evaluate the performance of our proposed method, we conduct validation experiments on standard public datasets and real robot grasping experiments. We achieve the image-wise detection accuracy of 99.4% and 96.7% on Cornell and Jacquard datasets respectively. The object-wise detection accuracy reaches 97.8% and 94.6% on the same datasets. Furthermore, physical experiments using the 6-DoF Elite robot demonstrate a success rate of 94.5%. These experiments highlight the superior accuracy of our proposed method.

show abstract

Section: Cornell Dataset Experiments Resultsmentioning

confidence: 99%

Section: Multiple Modality Fusion Based Grasp Detectionmentioning

confidence: 99%

Bilateral Cross-Modal Fusion Network for Robot Grasp Detection

Zhang¹,

Sun²,

Liu³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Morrison et al [14] used convolutional layers for encoding and decoding to perform pixel-level grasp prediction of feature maps. Yu et al [23] proposed a U-Net based neural network with channel attention modules to better utilize features. Wu et al [24] introduced an anchor-free grasp detector based on a fully convolutional network that formulates grasp detection as a closest horizontal or vertical rectangle regression task and a grasp angle classification task.…”

Section: Related Workmentioning

confidence: 99%

HTC-Grasp: A Hybrid Transformer-CNN Architecture for Robotic Grasp Detection

Zhang¹,

Jun²,

Sun³

et al. 2023

Preprint

View full text Add to dashboard Cite

We introduce a novel hybrid Transformer-CNN architecture for robotic grasp detection, designed to enhance the accuracy of grasping unknown objects. Our proposed architecture has two key designs. Firstly, we develop a hierarchical transformer as the encoder, incorporating the external attention to effectively capture the correlation features across the data. Secondly, the decoder is constructed with cross-layer connections to efficiently fuse multi-scale features. Channel attention is introduced in the decoder to model the correlation between channels and to adaptively recalibrate the channel correlation feature response, thereby increasing the weight of the effective channels. Our method is evaluated on the Cornell and Jacquard public datasets, achieving an image-wise detection accuracy of 98.3% and 95.8% on each dataset, respectively. Additionally, we achieve object-wise detection accuracy of 96.9% and 92.4% on the same datasets. A physical experiment is also performed using the Elite 6Dof robot, with a grasping accuracy rate of 93.3%, demonstrating the proposed method's ability to grasp unknown objects in real-world scenarios. The results of this study show that our proposed method outperforms other state-of-the-art methods.

show abstract

“…As a core component of autonomous grasping, grasp detection, which outputs the most possible grasp configuration for the manipulator, has attracted great attention from both academic and industrial communities. Existing methods often predict a series of possible grasp configurations based on the input images (Depierre et al, 2018 ; Zhang et al, 2019 ; Wang et al, 2021 ; Yu et al, 2022b ). When encountered with a cluttered scene, which is a common case in our daily life, we humans often identify the target object first and then determine the best pose to grab the object.…”

Section: Introductionmentioning

confidence: 99%

“…To overcome this short back, other studies (Park et al, 2020 ) use classification and regression processes to predict the final angles. Another kind of grasp detection method (Yu et al, 2022b ) makes dense predictions at each pixel and outputs a set of heatmaps representing the grasp configurations and quality.…”

Section: Introductionmentioning

confidence: 99%

A neural learning approach for simultaneous object detection and grasp detection in cluttered scenes

Yang

Xie²,

Li³

2023

Front. Comput. Neurosci.

View full text Add to dashboard Cite

Object detection and grasp detection are essential for unmanned systems working in cluttered real-world environments. Detecting grasp configurations for each object in the scene would enable reasoning manipulations. However, finding the relationships between objects and grasp configurations is still a challenging problem. To achieve this, we propose a novel neural learning approach, namely SOGD, to predict a best grasp configuration for each detected objects from an RGB-D image. The cluttered background is first filtered out via a 3D-plane-based approach. Then two separate branches are designed to detect objects and grasp candidates, respectively. The relationship between object proposals and grasp candidates are learned by an additional alignment module. A series of experiments are conducted on two public datasets (Cornell Grasp Dataset and Jacquard Dataset) and the results demonstrate the superior performance of our SOGD against SOTA methods in predicting reasonable grasp configurations “from a cluttered scene.”

show abstract

SE-ResUNet: A Novel Robotic Grasp Detection Method

Cited by 72 publications

References 33 publications

Bilateral Cross-Modal Fusion Network for Robot Grasp Detection

Bilateral Cross-Modal Fusion Network for Robot Grasp Detection

HTC-Grasp: A Hybrid Transformer-CNN Architecture for Robotic Grasp Detection

A neural learning approach for simultaneous object detection and grasp detection in cluttered scenes

Contact Info

Product

Resources

About