Improving Haptic Exploration of Object Shape by Discovering Symmetries

Bonzini, Aramis Augusto; Seminara, Lucia; Jamone, Lorenzo

doi:10.1109/icra46639.2022.9812200

Cited by 5 publications

(3 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, points covering most of the surface are needed, which leads to the need for a lot of exploration. A way to overcome this issue may be to use symmetry as in [19].…”

Section: Visuo-haptic Shape Completionmentioning

confidence: 99%

“…However, these are not as effective as an uncertainty-driven approach. Uncertainty can come from the Gaussian distribution [16]- [19], [21]; from the Monte Carlo dropout [24]; or from the Signed Distance Function (SDF) [1], [25]. Alternatively, it can be learned where to touch as in Smith et al [23].…”

Section: Visuo-haptic Shape Completionmentioning

confidence: 99%

“…After collision, new visual information is saved, segmented and added to the point cloud for the touched object, together with the haptic information (box (7), lines [15][16][17][18][19].…”

Section: H Vishac Algorithmmentioning

confidence: 99%

See 2 more Smart Citations

Efficient Visuo-Haptic Object Shape Completion for Robot Manipulation

Rustler¹,

Matas²,

Hoffmann³

2023

Preprint

View full text Add to dashboard Cite

For robot manipulation, a complete and accurate object shape is desirable. Here, we present a method that combines visual and haptic reconstruction in a closed-loop pipeline. From an initial viewpoint, the object shape is reconstructed using an implicit surface deep neural network. The location with highest uncertainty is selected for haptic exploration, the object is touched, the new information from touch and a new point cloud from the camera are added, object position is re-estimated and the cycle is repeated. We extend Rustler et al. (2022) by using a new theoretically grounded method to determine the points with highest uncertainty, and we increase the yield of every haptic exploration by adding not only the contact points to the point cloud but also incorporating the empty space established through the robot movement to the object. Additionally, the solution is compact in that the jaws of a closed two-finger gripper are directly used for exploration. The object position is re-estimated after every robot action and multiple objects can be present simultaneously on the table. We achieve a steady improvement with every touch using three different metrics and demonstrate the utility of the better shape reconstruction in grasping experiments on the real robot. On average, grasp success rate increases from 63.3% to 70.4% after a single exploratory touch and to 82.7% after five touches. The collected data and code are publicly available (https://osf. io/j6rkd/, https://github.com/ctu-vras/vishac).

show abstract

“…However, points covering most of the surface are needed, which leads to the need for a lot of exploration. A way to overcome this issue may be to use symmetry as in [19].…”

Section: Visuo-haptic Shape Completionmentioning

confidence: 99%