Keypoint-Based Robotic Grasp Detection Scheme in Multi-Object Scenes

Li, Tong; Wang, Fei; Ru, Changlei; Jiang, Yong; Li, Jinghong

doi:10.3390/s21062132

Cited by 18 publications

(15 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With the deepening of sEMG detection technology and the rapid development of computer technology, sEMG controlled human-machine interaction ( Sun et al, 2020a ; Chen et al, 2022c ) systems can analyze the sEMG generated during the user’s movement to obtain the human body’s intention, and eventually control peripheral devices by transmitting movement commands ( Pinto and Gupta. 2016 ; Zhang et al, 2019 ; Li et al, 2021 ; Liu et al, 2022a ). Early prostheses were generally single-degree-of-freedom robotic arms with grasping capability only.…”

Section: Related Workmentioning

confidence: 99%

Improved Multi-Stream Convolutional Block Attention Module for sEMG-Based Gesture Recognition

Wang

Huang

et al. 2022

Front. Bioeng. Biotechnol.

View full text Add to dashboard Cite

As a key technology for the non-invasive human-machine interface that has received much attention in the industry and academia, surface EMG (sEMG) signals display great potential and advantages in the field of human-machine collaboration. Currently, gesture recognition based on sEMG signals suffers from inadequate feature extraction, difficulty in distinguishing similar gestures, and low accuracy of multi-gesture recognition. To solve these problems a new sEMG gesture recognition network called Multi-stream Convolutional Block Attention Module-Gate Recurrent Unit (MCBAM-GRU) is proposed, which is based on sEMG signals. The network is a multi-stream attention network formed by embedding a GRU module based on CBAM. Fusing sEMG and ACC signals further improves the accuracy of gesture action recognition. The experimental results show that the proposed method obtains excellent performance on dataset collected in this paper with the recognition accuracies of 94.1%, achieving advanced performance with accuracy of 89.7% on the Ninapro DB1 dataset. The system has high accuracy in classifying 52 kinds of different gestures, and the delay is less than 300 ms, showing excellent performance in terms of real-time human-computer interaction and flexibility of manipulator control.

show abstract

Section: Related Workmentioning

confidence: 99%

Improved Multi-Stream Convolutional Block Attention Module for sEMG-Based Gesture Recognition

Wang

Huang

et al. 2022

Front. Bioeng. Biotechnol.

View full text Add to dashboard Cite

show abstract

“…Such method uses the complete 3D model of the target object to define the grasp operation. However, robots face different environments, and obtaining the accurate 3D model in advance seems to be impossible [19]. On the contrary, it is more convenient to capture RGB images than reconstructed 3D models.…”

Section: Grasp Detection Using Neural Networkmentioning

confidence: 99%

“…ese works pay no attention on multiobject, occluded cases and instruct the robot to grasp objects only under ideal circumstances [19]. Lenz et al [21] connected two single neural networks in series in order to detect grasp positions in an RGB-D image.…”

Section: Grasp Detection Using Neural Networkmentioning

confidence: 99%

Grasp Detection under Occlusions Using SIFT Features

Guo

Wang

et al. 2021

Complexity

View full text Add to dashboard Cite

Distinguishing target object under occlusions has become the forefront of research to cope with grasping study in general. In this paper, a novel framework which is able to be utilized for a parallel robotic gripper is proposed. There are two key steps for the proposed method in the process of grasping occluded object: generating template information and grasp detection using the matching algorithm. A neural network, trained by the RGB-D data from the Cornell Grasp Dataset, predicts multiple grasp rectangles on template images. A proposed matching algorithm is utilized to eliminate the influence caused by occluded parts on scene images and generates multiple grasp rectangles for objects under occlusions using the grasp information of matched template images. In order to improve the quality of matching result, the proposed matching algorithm improves the SIFT algorithm and combines it with the improved RANSAC algorithm. In this way, this paper obtains suitable grasp rectangles on scene images and offers a new thought about grasping detection under occlusions. The validation results show the effectiveness and efficiency of this approach.

show abstract

“…Object keypoints is an inexpensive way to describe object shapes and poses in grasping. In fact, many grasping detection and pose estimation works are based on object keypoints [32], [33], but the dataset used in these works are limited in terms of scale and class diversity. The proposed method to generate MetaGraspNet can be divided into three steps: putting together a diverse item set, sampling ambidextrous grasp labels for each object individually, generating bin scenes together with rich annotations in the metaverse (see Fig.…”

Section: Introductionmentioning

confidence: 99%

MetaGraspNet: A Large-Scale Benchmark Dataset for Scene-Aware Ambidextrous Bin Picking via Physics-based Metaverse Synthesis

Gilles¹,

Chen²,

Winter³

et al. 2022

Preprint

View full text Add to dashboard Cite

Autonomous bin picking poses significant challenges to vision-driven robotic systems given the complexity of the problem, ranging from various sensor modalities, to highly entangled object layouts, to diverse item properties and gripper types. Existing methods often address the problem from one perspective. Diverse items and complex bin scenes require diverse picking strategies together with advanced reasoning. As such, to build robust and effective machine-learning algorithms for solving this complex task requires significant amounts of comprehensive and high quality data. Collecting such data in real world would be too expensive and time prohibitive and therefore intractable from a scalability perspective. To tackle this big, diverse data problem, we take inspiration from the recent rise in the concept of metaverses, and introduce MetaGraspNet, a large-scale photo-realistic bin picking dataset constructed via physics-based metaverse synthesis. The proposed dataset contains 217k RGBD images across 82 different article types, with full annotations for object detection, amodal perception, keypoint detection, manipulation order and ambidextrous grasp labels for a parallel-jaw and vacuum gripper. We also provide a real dataset consisting of over 2.3k fully annotated high-quality RGBD images, divided into 5 levels of difficulties and an unseen object set to evaluate different object and layout properties. Finally, we conduct extensive experiments showing that our proposed vacuum seal model and synthetic dataset achieves state-of-the-art performance and generalizes to real world use-cases.

show abstract

Keypoint-Based Robotic Grasp Detection Scheme in Multi-Object Scenes

Cited by 18 publications

References 30 publications

Improved Multi-Stream Convolutional Block Attention Module for sEMG-Based Gesture Recognition

Improved Multi-Stream Convolutional Block Attention Module for sEMG-Based Gesture Recognition

Grasp Detection under Occlusions Using SIFT Features

MetaGraspNet: A Large-Scale Benchmark Dataset for Scene-Aware Ambidextrous Bin Picking via Physics-based Metaverse Synthesis

Contact Info

Product

Resources

About