Learning robust, real-time, reactive robotic grasping

Abstract

We present a novel approach to perform object-independent grasp synthesis from depth images via deep neural networks. Our generative grasping convolutional neural network (GG-CNN) predicts a pixel-wise grasp quality that can be deployed in closed-loop grasping scenarios. GG-CNN overcomes shortcomings in existing techniques, namely discrete sampling of grasp candidates and long computation times. The network is orders of magnitude smaller than other state-of-the-art approaches while achieving better performance, particularly in clutter. We run a suite of real-world tests, during which we achieve an 84% grasp success rate on a set of previously unseen objects with adversarial geometry and 94% on household items. The lightweight nature enables closed-loop control of up to 50 Hz, with which we observed 88% grasp success on a set of household objects that are moved during the grasp attempt. We further propose a method combining our GG-CNN with a multi-view approach, which improves overall grasp success rate in clutter by 10%. Code is provided at https://github.com/dougsm/ggcnn

Robotic grasp detection using deep convolutional neural networks

“…Grasping using Uni-modal data : Johns et al [25] used a simulated depth image to predict a grasp outcome for every grasp pose predicted and select the best grasp by smoothing the predicted pose using a grasp uncertainty function. A generative approach to grasping is discussed by Morrison et al [26]. The Generative grasp CNN architecture generates grasp poses using a depth image and the network computes grasp on a pixel-wise basis and insists that it reduces existing shortcomings of discrete sampling and computational complexity.…”

“…Instead of the 5 dimensional grasp representation used in [1], [2], [32], we use an improved version of grasp representation similar to the one proposed by Morrison et al in [26]. We denote the grasp pose in robot frame as:…”

Multi-Modal Transfer Learning for Grasping Transparent and Specular Objects

“…Empirical approaches, on the other hand, learn to predict the quality of grasp candidates from data on a diverse set of objects, images, and grasp attempts collected through human labeling [19], [20], [21], [22], self-supervision [23], [24], or simulated data [25], [26], [3], [27], [1]. Saxena et al [19] trained a classifier on human-labeled RGB images to predict grasp points, triangulated the points on stereo RGB images, and demonstrated successful grasps on a limited set of household objects, including some transparent and specular objects.…”

“…Recently, approaches trained on data gathered in simulation have demonstrated state-of-the-art performance. The Jacquard dataset by Amaury et al [25] uses a grasp specification similar to the Cornell Grasping Dataset, contains simulated objects and grasp attempts, and has been successfully used for training by Morrison et al 's GG-CNN [26]. Mahler et al [27] developed GQCNN, which was trained on a dataset of simulated grasps generated using analytic model, representing a hybrid empirical and analytic approach.…”

Object Detection-Based One-Shot Imitation Learning with an RGB-D Camera

“…Chen et al trained the visuomotor policy with depth information and semantic information in simulation and transferred it to the real world for robotic navigation [32]. Morrison et al learned grasping skills with synthetic depth maps and tested in the real world [33].…”