DVGG: Deep Variational Grasp Generation for Dextrous Manipulation

Abstract

Grasping with anthropomorphic robotic hands involves much more hand-object interactions compared to paralleljaw grippers. Modeling hand-object interactions is essential to the study of multi-finger hand dextrous manipulation. This work presents DVGG, an efficient grasp generation network that takes single-view observation as input and predicts high-quality grasp configurations for unknown objects. In general, our generative model consists of three components: 1) Point cloud completion for the target object based on the partial observation; 2) Diverse sets of grasps generation given the complete point cloud; 3) Iterative grasp pose refinement for physically plausible grasp optimization. To train our model, we build a large-scale grasping dataset that contains about 300 common object models with 1.5M annotated grasps in simulation. Experiments in simulation show that our model can predict robust grasp poses with a wide variety and high success rate. Real robot platform experiments demonstrate that the model trained on our dataset performs well in the real world. Remarkably, our method achieves a grasp success rate of 70.7% for novel objects in the real robot platform, which is a significant improvement over the baseline methods.

A 6DOF pose measurement method for metal casts object based on stereo vision sensor

“…Le put forward a robot 2D positioning algorithm based on depth learning in literature (Le and Lin, 2019), which realized disorderly grasping of specific kinds of workpieces. Literature (Wei et al , 2022) combines 6DOF cooperative robot with bionic hand, studies the system’s grasp of target objects through deep neural network and explores the development direction of future robots. However, when the object detection algorithm based on deep learning is applied to industrial objects, the positioning accuracy of the system is not high, which restricts the promotion of the deep learning algorithm in the industrial field.…”

DexRepNet: Learning Dexterous Robotic Grasping Network with Geometric and Spatial Hand-Object Representations

“…Shao [7] models the robotic hand and object and uses optimization methods to find force closure and contact points. Wei et al [6] presents an efficient grasp generation network that takes singleview point cloud reconstructed by point completion module as input and predicts high-quality grasp configurations for unknown objects. Turpin et al [14] and Liu et al [15] adopt differentiable simulation to optimize a path towards stable grasping.…”

“…Hand-Object Representation for Grasping DRL methods emphasize self-exploration in the interaction with the environment, therefore, a detailed description of the interaction is crucial. For object description, there are RGB [10], depth [15], point cloud [6], mesh [17], and their fusion [3], [4], [18]. The description for hand is mainly the hand state [3], [4].…”

“…However, less attention has been paid to hand-object interaction representations for high-level generalization with dexterous hands. For point cloud inputs, features such as position, distance, or/and a global feature vector extracted by PointNet [5] describe the relationship between hands and objects [6], [7], [8]; for input with RGB images, a global feature vector extracted from CNN like ResNet [9] is used to capture the relation [10]. Though able to capture the geometry of objects, global features typically have difficulty in generalizing to new data.…”

Dexterous Manipulation for Multi-Fingered Robotic Hands With Reinforcement Learning: A Review

“…Recently, the power of artificial intelligence has attracted the attention of many researchers. Deep learning has even reached a level that exceeds that of humans in certain fields, such as computer vision, so the robot can extract generalized features autonomously (LeCun et al, 2015 ; Duan et al, 2021 ; Wei et al, 2021 , 2022 ; Li et al, 2022 ). Deep learning is better at classification and prediction problems and so on.…”