DGCM-Net: Dense Geometrical Correspondence Matching Network for Incremental Experience-Based Robotic Grasping

Abstract

This article presents a method for grasping novel objects by learning from experience. Successful attempts are remembered and then used to guide future grasps such that more reliable grasping is achieved over time. To transfer the learned experience to unseen objects, we introduce the dense geometric correspondence matching network (DGCM-Net). This applies metric learning to encode objects with similar geometry nearby in feature space. Retrieving relevant experience for an unseen object is thus a nearest neighbor search with the encoded feature maps. DGCM-Net also reconstructs 3D-3D correspondences using the view-dependent normalized object coordinate space to transform grasp configurations from retrieved samples to unseen objects. In comparison to baseline methods, our approach achieves an equivalent grasp success rate. However, the baselines are significantly improved when fusing the knowledge from experience with their grasp proposal strategy. Offline experiments with a grasping dataset highlight the capability to transfer grasps to new instances as well as to improve success rate over time from increasing experience. Lastly, by learning task-relevant grasps, our approach can prioritize grasp configurations that enable the functional use of objects.

CaTGrasp: Learning Category-Level Task-Relevant Grasping in Clutter from Simulation

“…Compared to the original NOCS representation [11], which recovers a 7D pose of novel object instances, the proposed NUNOCS allows to scale independently in each dimension when converting to the canonical space. Therefore, more fine-grained dense correspondence across object instances can be established via measuring their similarity (𝐿 2 distance in our case) in C. This is especially the case for instances with dramatically different 3D scales, as shown in the wrapped figure, where colors indicate correspondence similarity in C. A key difference from another related work on VD-NOC [44], which directly normalizes the scanned point cloud in the camera frame, is that the proposed NUNOCS representation is object-centric and thus agnostic to specific camera parameters or viewpoints.…”

TransGrasp: Grasp Pose Estimation of a Category of Objects by Transferring Grasps from Only One Labeled Instance

“…Closely related to our work, some methods transfer grasps to other objects by building dense correspondence. DGCM-Net [19] transfers grasps by predicting view-dependent normalized object coordinate (VD-NOC) values between pairs of depth images. CaTGrasp [29] maps the input point cloud to a Non-Uniform Normalized Object Coordinate Space (NUNOCS) where the spatial correspondence is built.…”

Optimal Grasping Strategy for Robots With a Parallel Gripper Based on Feature Sensing of 3D Object Model

“…As a result, the grasp can be transformed from the same category's object. DGCM-Net [24] adopted the network to learn a reliable grasp, and then transferred the grasp to unseen objects in the same category. In order to transfer the grasp, both methods need to detect the object and separate the point cloud of the object from the depth map first.…”