DDGC: Generative Deep Dexterous Grasping in Clutter

Lundell, Jens; Verdoja, Francesco; Kyrki, Ville

doi:10.1109/lra.2021.3096239

Cited by 46 publications

(26 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The detailed predictions for each object include its 3D model, all naturally possible hand grasp types defined in Feix et al ( 2016 ) and the corresponding refined 51-DoF 3D hand models by minimizing a graspability loss. Generative Deep Dexterous Grasping in Clutter (DDGC) method (Lundell et al, 2021 ) has a similar structure as GANhand, but it adds depth channel and directly.…”

Section: Learning-based Manipulation Methodsmentioning

confidence: 99%

A Survey of Multifingered Robotic Manipulation: Biological Results, Structural Evolvements, and Learning Methods

Wang

et al. 2022

Front. Neurorobot.

View full text Add to dashboard Cite

Multifingered robotic hands (usually referred to as dexterous hands) are designed to achieve human-level or human-like manipulations for robots or as prostheses for the disabled. The research dates back 30 years ago, yet, there remain great challenges to effectively design and control them due to their high dimensionality of configuration, frequently switched interaction modes, and various task generalization requirements. This article aims to give a brief overview of multifingered robotic manipulation from three aspects: a) the biological results, b) the structural evolvements, and c) the learning methods, and discuss potential future directions. First, we investigate the structure and principle of hand-centered visual sensing, tactile sensing, and motor control and related behavioral results. Then, we review several typical multifingered dexterous hands from task scenarios, actuation mechanisms, and in-hand sensors points. Third, we report the recent progress of various learning-based multifingered manipulation methods, including but not limited to reinforcement learning, imitation learning, and other sub-class methods. The article concludes with open issues and our thoughts on future directions.

show abstract

Section: Learning-based Manipulation Methodsmentioning

confidence: 99%

A Survey of Multifingered Robotic Manipulation: Biological Results, Structural Evolvements, and Learning Methods

Wang

et al. 2022

Front. Neurorobot.

View full text Add to dashboard Cite

show abstract

“…Kokic et al [54] randomly sample grasp and roll angles, and offset distances for each point in the point cloud. Both Lu et al [43], Lundell et al [55,57] sample grasp candidates around the center of an object with a random orientation.…”

Section: A Samplingmentioning

confidence: 99%

“…Similarly, Ottenhaus et al [44] train a CNN to estimate the force-closure probability of a grasp under a small random perturbation. Lundell et al [55] and Lundell et al [57] trained a grasp classifier using a Generative Adversarial Networks (GANs), and use the discriminator loss to help produce realistic-looking grasps. Wen et al [65] compute a continuous score for each grasp by looking at the stability of randomly sampled grasps in the proximity of the selected grasp.…”

Section: A Samplingmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning Approaches to Grasp Synthesis: A Review

Newbury¹,

Gu²,

Lachlan³

et al. 2022

Preprint

View full text Add to dashboard Cite

Grasping is the process of picking an object by applying forces and torques at a set of contacts. Recent advances in deep-learning methods have allowed rapid progress in robotic object grasping. We systematically surveyed the publications over the last decade, with a particular interest in grasping an object using all 6 degrees of freedom of the end-effector pose. Our review found four common methodologies for robotic grasping: sampling-based approaches, direct regression, reinforcement learning, and exemplar approaches. Furthermore, we found two 'supporting methods' around grasping that use deeplearning to support the grasping process, shape approximation, and affordances. We have distilled the publications found in this systematic review (85 papers) into ten key takeaways we consider crucial for future robotic grasping and manipulation research.

show abstract

“…[10]). More recent papers have shifted to complex model architectures such as generative models [13,14] and convolutional neural networks [15,16] to produce grasps directly. The main limitation of these works lies in restricted hardware evaluation.…”

Section: Simple and Dexterous Grasping In Literaturementioning

confidence: 99%

Learning Diverse and Physically Feasible Dexterous Grasps with Generative Model and Bilevel Optimization

Wu¹,

Guo²,

Liu³

2022

Preprint

View full text Add to dashboard Cite

To fully utilize the versatility of a multi-finger dexterous robotic hand for object grasping, one must satisfy complex physical constraints introduced by hand-object interaction and object geometry during grasp planning. We propose an integrative approach of combining a generative model and a bilevel optimization to compute diverse grasps for novel unseen objects. First, a grasp prediction is obtained from a conditional variational autoencoder trained on merely six YCB objects. The prediction is then projected onto the manifold of kinematically and dynamically feasible grasps by jointly solving collision-aware inverse kinematics, force closure, and friction constraints as one nonconvex bilevel optimization. We demonstrate the effectiveness of our method on hardware by successfully grasping a wide range of unseen household objects, including adversarial shapes challenging to other types of robotic grippers. A video summary of our results is available at https://youtu.be/9DTrImbN99I.

show abstract

DDGC: Generative Deep Dexterous Grasping in Clutter

Cited by 46 publications

References 33 publications

A Survey of Multifingered Robotic Manipulation: Biological Results, Structural Evolvements, and Learning Methods

A Survey of Multifingered Robotic Manipulation: Biological Results, Structural Evolvements, and Learning Methods

Deep Learning Approaches to Grasp Synthesis: A Review

Learning Diverse and Physically Feasible Dexterous Grasps with Generative Model and Bilevel Optimization

Contact Info

Product

Resources

About