Active perception: Building objects' models using tactile exploration

Jamali, Nawid; Ciliberto, Carlo; Rosasco, Lorenzo; Natale, Lorenzo

doi:10.1109/humanoids.2016.7803275

Cited by 52 publications

(35 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is extensive work based on tactile sensing that aims at recovering the shape of manipulated objects [15,16,17,18,19,20,21,22,23]. Among the most recent works, Jamali et al [18] and Driess et al [20] actively search for regions that need to be explored based on tactile readings. However, their sensors are low resolution which makes the reconstructed shapes hard to use for other applications.…”

Section: Related Workmentioning

confidence: 99%

Tactile Mapping and Localization from High-Resolution Tactile Imprints

Bauzá

Canal

Rodríguez

2019

2019 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

This work studies the problem of shape reconstruction and object localization using a vision-based tactile sensor, GelSlim. The main contributions are the recovery of local shapes from contact, an approach to reconstruct the tactile shape of objects from tactile imprints, and an accurate method for object localization of previously reconstructed objects. The algorithms can be applied to a large variety of 3D objects and provide accurate tactile feedback for in-hand manipulation.Results show that by exploiting the dense tactile information we can reconstruct the shape of objects with high accuracy and do on-line object identification and localization, opening the door to reactive manipulation guided by tactile sensing. We provide videos and supplemental information in the project's website web.mit.edu/mcube/research/tactile localization.html.

show abstract

Section: Related Workmentioning

confidence: 99%

Tactile Mapping and Localization from High-Resolution Tactile Imprints

Bauzá

Canal

Rodríguez

2019

2019 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

show abstract

“…For shape reconstruction in particular, tactile data have also been exploited for both local [28] [29] and global shape completion [30], [31], sometimes in a bimanual setting [32]. In recent years, researchers started to use active learning for shape reconstruction from tactile sensing [33], [34], [35], [36]. Luo et al recently wrote a comprehensive review article on tactile perception which includes object shape perception [37].…”

Section: B Tactile Sensing For Shape Reconstructionmentioning

confidence: 99%

3D Shape Perception from Monocular Vision, Touch, and Shape Priors

Wang

Sun

et al. 2018

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

104

View full text Add to dashboard Cite

Perceiving accurate 3D object shape is important for robots to interact with the physical world. Current research along this direction has been primarily relying on visual observations. Vision, however useful, has inherent limitations due to occlusions and the 2D-3D ambiguities, especially for perception with a monocular camera. In contrast, touch gets precise local shape information, though its efficiency for reconstructing the entire shape could be low. In this paper, we propose a novel paradigm that efficiently perceives accurate 3D object shape by incorporating visual and tactile observations, as well as prior knowledge of common object shapes learned from large-scale shape repositories. We use vision first, applying neural networks with learned shape priors to predict an object's 3D shape from a single-view color image. We then use tactile sensing to refine the shape; the robot actively touches the object regions where the visual prediction has high uncertainty. Our method efficiently builds the 3D shape of common objects from a color image and a small number of tactile explorations (around 10). Our setup is easy to apply and has potentials to help robots better perform grasping or manipulation tasks on real-world objects.

show abstract

“…Early work in the field focussed on using passive stereo vision and active touch to explore object contours and features [1]- [3]. Following work has combined touch and 3D vision to rapidly label surface features [17], map object surfaces [18], match tactile features to visual maps [19] and nest visual and tactile control loops to improve surface exploration [20], [21].…”

Section: Background and Related Workmentioning

confidence: 99%