Team Delft's Robot Winner of the Amazon Picking Challenge 2016

Herńandez, C. F. Gonźalez; Bharatheesha, Mukunda; Ko, Wilson; Gaiser, Hans; Tan, Jethro; Deurzen, Kanter van; Vries, Maarten de; Mil, Bas Van; Egmond, Jeff van; Burger, Ruben; Morariu, M.; Ju, Jonghyun; Gerrmann, Xander; Ensing, Ronald; Frankenhuyzen, Jan van; Wisse, Martijn

doi:10.48550/arxiv.1610.05514

Cited by 9 publications

(16 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Point-Cloud Registration (REG). We also compared with grasp planning based on point cloud registration, a state-ofthe-art method for using precomputed grasps [13,20]. We first coarsely estimated the object instance and pose based on the top 3 most similar synthetic images from Dex-Net 2.0, where similarity is measured as distance between AlexNet conv5 features [13,34].…”

Section: Grasp Planning Methods Used For Comparisonmentioning

confidence: 99%

“…To execute grasps on a physical robot, a common approach is to precompute a database of known 3D objects labeled with grasps and quality metrics such as GraspIt! [14].Precomputed grasps are indexed using point cloud registration: matching point clouds to known 3D object models in the database using visual and geometric similarity [4,5,7,13,20,22,27] and executing the highest quality grasp for the estimated object instance.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics

Mahler¹,

Liang²,

Niyaz³

et al. 2017

Robotics: Science and Systems XIII

1,021

936

View full text Add to dashboard Cite

Abstract-To reduce data collection time for deep learning of robust robotic grasp plans, we explore training from a synthetic dataset of 6.7 million point clouds, grasps, and analytic grasp metrics generated from thousands of 3D models from Dex-Net 1.0 in randomized poses on a table. We use the resulting dataset, DexNet 2.0, to train a Grasp Quality Convolutional Neural Network (GQ-CNN) model that rapidly predicts the probability of success of grasps from depth images, where grasps are specified as the planar position, angle, and depth of a gripper relative to an RGB-D sensor. Experiments with over 1,000 trials on an ABB YuMi comparing grasp planning methods on singulated objects suggest that a GQ-CNN trained with only synthetic data from Dex-Net 2.0 can be used to plan grasps in 0.8s with a success rate of 93% on eight known objects with adversarial geometry and is 3× faster than registering point clouds to a precomputed dataset of objects and indexing grasps. The Dex-Net 2.0 grasp planner also has the highest success rate on a dataset of 10 novel rigid objects and achieves 99% precision (one false positive out of 69 grasps classified as robust) on a dataset of 40 novel household objects, some of which are articulated or deformable. Code, datasets, videos, and supplementary material are available at http://berkeleyautomation.github.io/dex-net.

show abstract

Section: Grasp Planning Methods Used For Comparisonmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics

Mahler¹,

Liang²,

Niyaz³

et al. 2017

Robotics: Science and Systems XIII

1,021

936

View full text Add to dashboard Cite

show abstract

“…In 2015, Team RBO [6] won by pushing objects from the top or side until suction was achieved, and Team MIT [35] came in second place by suctioning on the centroid of objects with flat surfaces. In 2016, Team Delft [10] won the challenge by approaching the estimated object centroid along the inward surface normal. In 2017, Cartman [?]…”

Section: B Grasp Planningmentioning

confidence: 99%

“…While grasp planning searches for gripper configurations that maximize a quality metric derived from mechanical wrench space analysis [24], human labels [28], or selfsupervised labels [20], suction grasps are often planned directly on point clouds using heuristics such as grasping near the object centroid [10] or at the center of planar surfaces [4], [5]. These heuristics work well for prismatic objects such as boxes and cylinders but may fail on objects with non-planar surfaces near the object centroid, which is common for industrial parts and household objects such as staplers or children's toys.…”

Section: Introductionmentioning

confidence: 99%

Dex-Net 3.0: Computing Robust Vacuum Suction Grasp Targets in Point Clouds Using a New Analytic Model and Deep Learning

Mahler¹,

Matl²,

Liu³

et al. 2018

2018 IEEE International Conference on Robotics and Automation (ICRA)

269

199

View full text Add to dashboard Cite

Vacuum-based end effectors are widely used in industry and are often preferred over parallel-jaw and multifinger grippers due to their ability to lift objects with a single point of contact. Suction grasp planners often target planar surfaces on point clouds near the estimated centroid of an object. In this paper, we propose a compliant suction contact model that computes the quality of the seal between the suction cup and local target surface and a measure of the ability of the suction grasp to resist an external gravity wrench. To characterize grasps, we estimate robustness to perturbations in end-effector and object pose, material properties, and external wrenches. We analyze grasps across 1,500 3D object models to generate Dex-Net 3.0, a dataset of 2.8 million point clouds, suction grasps, and grasp robustness labels. We use Dex-Net 3.0 to train a Grasp Quality Convolutional Neural Network (GQ-CNN) to classify robust suction targets in point clouds containing a single object. We evaluate the resulting system in 350 physical trials on an ABB YuMi fitted with a pneumatic suction gripper. When evaluated on novel objects that we categorize as Basic (prismatic or cylindrical), Typical (more complex geometry), and Adversarial (with few available suction-grasp points) Dex-Net 3.0 achieves success rates of 98%, 82%, and 58% respectively, improving to 81% in the latter case when the training set includes only adversarial objects. Code, datasets, and supplemental material can be found at http://berkeleyautomation.github.io/dex-net.

show abstract

“…accurate pose estimation for the entire scene. In this domain, solutions have been developed that use a Convolutional Neural Network (CNN) for object segmentation [2], [3] followed by a 3D model alignment step using point cloud registration techniques for pose estimation [4], [5]. The focus of the current paper is to improve this last step and increase the accuracy of pose estimation by reasoning at a scene-level about the physical interactions between objects.…”

Section: Introductionmentioning

confidence: 99%

Improving 6D Pose Estimation of Objects in Clutter Via Physics-Aware Monte Carlo Tree Search

Mitash

Boularias

Bekris

2018

2018 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

This work proposes a process for efficiently searching over combinations of individual object 6D pose hypotheses in cluttered scenes, especially in cases involving occlusions and objects resting on each other. The initial set of candidate object poses is generated from state-of-the-art object detection and global point cloud registration techniques. The best scored pose per object by using these techniques may not be accurate due to overlaps and occlusions. Nevertheless, experimental indications provided in this work show that object poses with lower ranks may be closer to the real poses than ones with high ranks according to registration techniques. This motivates a global optimization process for improving these poses by taking into account scene-level physical interactions between objects. It also implies that the Cartesian product of candidate poses for interacting objects must be searched so as to identify the best scene-level hypothesis. To perform the search efficiently, the candidate poses for each object are clustered so as to reduce their number but still keep a sufficient diversity. Then, searching over the combinations of candidate object poses is performed through a Monte Carlo Tree Search (MCTS) process that uses the similarity between the observed depth image of the scene and a rendering of the scene given the hypothesized pose as a score that guides the search procedure. MCTS handles in a principled way the tradeoff between fine-tuning the most promising poses and exploring new ones, by using the Upper Confidence Bound (UCB) technique. Experimental results indicate that this process is able to quickly identify in cluttered scenes physically-consistent object poses that are significantly closer to ground truth compared to poses found by point cloud registration methods.

show abstract

Team Delft's Robot Winner of the Amazon Picking Challenge 2016

Cited by 9 publications

References 0 publications

Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics

Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics

Dex-Net 3.0: Computing Robust Vacuum Suction Grasp Targets in Point Clouds Using a New Analytic Model and Deep Learning

Improving 6D Pose Estimation of Objects in Clutter Via Physics-Aware Monte Carlo Tree Search

Contact Info

Product

Resources

About