Grasp’D: Differentiable Contact-Rich Grasp Synthesis for Multi-Fingered Hands

Turpin, Dylan; Wang, Liquan; Heiden, Eric; Chen, Yun-Chun; Macklin, Miles; Tsogkas, Stavros; Dickinson, Sven; Garg, Animesh

doi:10.1007/978-3-031-20068-7_12

Cited by 32 publications

(4 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some approaches (Lin et al 2023;Doosti et al 2020;Hasson et al 2019) directly estimate hand-object pose or reconstruct 3D mesh from a single image. Given the information of objects, several efforts (Jiang et al 2021;Brahmbhatt et al 2019;Turpin et al 2022) generate the corresponding hand-grasping pose. Furthermore, other approaches focus on refining hand-object grasping state (Grady et al 2021;Yang et al 2021;Tse et al 2022;Taheri et al 2020).…”

Section: Related Work Hand Object Interactionmentioning

confidence: 99%

Hand-Centric Motion Refinement for 3D Hand-Object Interaction via Hierarchical Spatial-Temporal Modeling

Hao,

Zhang,

Zhuo

et al. 2024

AAAI

View full text Add to dashboard Cite

Hands are the main medium when people interact with the world. Generating proper 3D motion for hand-object interaction is vital for applications such as virtual reality and robotics. Although grasp tracking or object manipulation synthesis can produce coarse hand motion, this kind of motion is inevitably noisy and full of jitter. To address this problem, we propose a data-driven method for coarse motion refinement. First, we design a hand-centric representation to describe the dynamic spatial-temporal relation between hands and objects. Compared to the object-centric representation, our hand-centric representation is straightforward and does not require an ambiguous projection process that converts object-based prediction into hand motion. Second, to capture the dynamic clues of hand-object interaction, we propose a new architecture that models the spatial and temporal structure in a hierarchical manner. Extensive experiments demonstrate that our method outperforms previous methods by a noticeable margin.

show abstract

Section: Related Work Hand Object Interactionmentioning

confidence: 99%

Hand-Centric Motion Refinement for 3D Hand-Object Interaction via Hierarchical Spatial-Temporal Modeling

Hao,

Zhang,

Zhuo

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Wei et al [6] presents an efficient grasp generation network that takes singleview point cloud reconstructed by point completion module as input and predicts high-quality grasp configurations for unknown objects. Turpin et al [14] and Liu et al [15] adopt differentiable simulation to optimize a path towards stable grasping. Deep reinforcement learning methods have gained increasing attention in recent years.…”

Section: Related Workmentioning

confidence: 99%

DexRepNet: Learning Dexterous Robotic Grasping Network with Geometric and Spatial Hand-Object Representations

Liu¹,

Chen²,

Sun³

et al. 2023

Preprint

View full text Add to dashboard Cite

Robotic dexterous grasping is a challenging problem due to the high degree of freedom (DoF) and complex contacts of multi-fingered robotic hands. Existing deep reinforcement learning (DRL) based methods leverage human demonstrations to reduce sample complexity due to the high dimensional action space with dexterous grasping. However, less attention has been paid to hand-object interaction representations for high-level generalization. In this paper, we propose a novel geometric and spatial hand-object interaction representation, named DexRep, to capture dynamic object shape features and the spatial relations between hands and objects during grasping. DexRep comprises Occupancy Feature for rough shapes within sensing range by moving hands, Surface Feature for changing hand-object surface distances, and Local-Geo Feature for local geometric surface features most related to potential contacts. Based on the new representation, we propose a dexterous deep reinforcement learning method to learn a generalizable grasping policy DexRepNet. Experimental results show that our method outperforms baselines using existing representations for robotic grasping dramatically both in grasp success rate and convergence speed. It achieves a 93% grasping success rate on seen objects and higher than 80% grasping success rates on diverse objects of unseen categories in both simulation and real-world experiments.

show abstract

“…In our approach, we use a non-learning based solution and hence do not suffer from generalization issues. Non-learningbased methods have been employed to generate extensive synthetic datasets [12], [14], [15], [16], [17]. [12] employs collision detection algorithms to formulate stable grasps.…”

Section: Related Work a Grasp Synthesis For Dexterous Handsmentioning

confidence: 99%

“…In a different vein, some approaches employ differentiable force closure estimation [14], [18] for grasp generation. Works such as [15], [16] exploit a differentiable simulation to synthesize grasps. In contrast to these works, our method can be conditioned on the grasp direction and does not rely on any simulator or force closure estimation.…”

Section: Related Work a Grasp Synthesis For Dexterous Handsmentioning

confidence: 99%

Learning Human-to-Robot Handovers from Point Clouds

Christen,

Yang,

Pérez-D'Arpino

et al. 2023

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Vision-based human-to-robot handover is an important and challenging task in human-robot interaction. Recent work has attempted to train robot policies by interacting with dynamic virtual humans in simulated environments, where the policies can later be transferred to the real world. However, a major bottleneck is the reliance on human motion capture data, which is expensive to acquire and difficult to scale to arbitrary objects and human grasping motions. In this paper, we introduce a framework that can generate plausible human grasping motions suitable for training the robot. To achieve this, we propose a hand-object synthesis method that is designed to generate handover-friendly motions similar to humans. This allows us to generate synthetic training and testing data with 100x more objects than previous work. In our experiments, we show that our method trained purely with synthetic data is competitive with state-of-the-art methods that rely on real human motion data both in simulation and on a real system. In addition, we can perform evaluations on a larger scale compared to prior work. With our newly introduced test set, we show that our model can better scale to a large variety of unseen objects and human motions compared to the baselines.

show abstract

Grasp’D: Differentiable Contact-Rich Grasp Synthesis for Multi-Fingered Hands

Cited by 32 publications

References 69 publications

Hand-Centric Motion Refinement for 3D Hand-Object Interaction via Hierarchical Spatial-Temporal Modeling

Hand-Centric Motion Refinement for 3D Hand-Object Interaction via Hierarchical Spatial-Temporal Modeling

DexRepNet: Learning Dexterous Robotic Grasping Network with Geometric and Spatial Hand-Object Representations

Learning Human-to-Robot Handovers from Point Clouds

Contact Info

Product

Resources

About