Finding safe 3D robot grasps through efficient haptic exploration with unscented Bayesian optimization and collision penalty

Abstract

Robust grasping is a major, and still unsolved, problem in robotics. Information about the 3D shape of an object can be obtained either from prior knowledge (e.g., accurate models of known objects or approximate models of familiar objects) or real-time sensing (e.g., partial point clouds of unknown objects) and can be used to identify good potential grasps. However, due to modeling and sensing inaccuracies, local exploration is often needed to refine such grasps and successfully apply them in the real world. The recently proposed unscented Bayesian optimization technique can make such exploration safer by selecting grasps that are robust to uncertainty in the input space (e.g., inaccuracies in the grasp execution). Extending our previous work on 2D optimization, in this paper we propose a 3D haptic exploration strategy that combines unscented Bayesian optimization with a novel collision penalty heuristic to find safe grasps in a very efficient way: while by augmenting the search-space to 3D we are able to find better grasps, the collision penalty heuristic allows us to do so without increasing the number of exploration steps.

Grasp Stability Prediction for a Dexterous Robotic Hand Combining Depth Vision and Haptic Bayesian Exploration

“…We extend the simulation results previously obtained in ( Nogueira et al, 2016 ; Castanheira et al, 2018 ) by testing the system with a real robot hand, and by performing experiments on three objects with complex shapes. Notably, many additional uncertainties are present in a real-world environment (e.g.…”

Visuo-Haptic Object Perception for Robots: An Overview

“…Finally, a haptic exploration procedure was performed, in which the hand touched the object several times with different tentative grasps, without lifting it, while evaluating a force closure grasp metric at each attempt. To reduce the number of exploration steps, the haptic exploration was realized with unscented bayesian optimization (Nogueira et al, 2016;Castanheira et al, 2018). Unscented bayesian optimization outperformed both bayesian optimization and random exploration (i.e., uniform grid search).…”

Simultaneous Tactile Exploration and Grasp Refinement for Unknown Objects

“…After initialisation, the process starts with the acquisition function suggesting a new point to explore. Our work differs from previous work using BO for grasp planning [33], [45] in that we combine visual and tactile sensing to reconstruct object surface while searching for grasp configurations and considering uncertainties. We perform exploration using all degrees of freedom of the hand, not only the wrist position, which is done in the robot's 6D task-space.…”