Object exploration using vision and active touch

Yang, Chuanyu; Lepora, Nathan F.

doi:10.1109/iros.2017.8206542

Cited by 10 publications

(6 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Taking advantage of the similarity between tactile and visual images, the same authors proposed algorithms to merge tactile and visual feedback for object localization and classification (Liu et al, 2017;Luo et al, 2015a). The combination of tactile and visual feedback has also been exploited by Yang and Lepora (2017) to implement an object exploration strategy.…”

Section: Contact Shape Processing and Classification With Tactile Imagesmentioning

confidence: 99%

Pressure distribution classification and segmentation of human hands in contact with the robot body

Albini

Cannata

2020

The International Journal of Robotics Research

View full text Add to dashboard Cite

This article deals with the problem of the recognition of human hand touch by a robot equipped with large area tactile sensors covering its body. This problem is relevant in the domain of physical human–robot interaction for discriminating between human and non-human contacts and to trigger and to drive cooperative tasks or robot motions, or to ensure a safe interaction. The underlying assumption used in this article is that voluntary physical interaction tasks involve hand touch over the robot body, and therefore the capability to recognize hand contacts is a key element to discriminate a purposive human touch from other types of interaction. The proposed approach is based on a geometric transformation of the tactile data, formed by pressure measurements associated to a non-uniform cloud of 3D points ( taxels) spread over a non-linear manifold corresponding to the robot body, into tactile images representing the contact pressure distribution in two dimensions. Tactile images can be processed using deep learning algorithms to recognize human hands and to compute the pressure distribution applied by the various hand segments: palm and single fingers. Experimental results, performed on a real robot covered with robot skin, show the effectiveness of the proposed methodology. Moreover, to evaluate its robustness, various types of failures have been simulated. A further analysis concerning the transferability of the system has been performed, considering contacts occurring on a different sensorized robot part.

show abstract

Section: Contact Shape Processing and Classification With Tactile Imagesmentioning

confidence: 99%

Pressure distribution classification and segmentation of human hands in contact with the robot body

Albini

Cannata

2020

The International Journal of Robotics Research

View full text Add to dashboard Cite

show abstract

“…We have the robot poke the object for once to detect the contact and update the local profiles of transparent objects, which is different from the previous tactile exploration works that contacts the object multiple times [41], [42]. Tactile exploration can be used to estimate the object shape so as to facilitate grasping.…”

Section: Conclusion and Discussionmentioning

confidence: 99%

Where Shall I Touch? Vision-Guided Tactile Poking for Transparent Object Grasping

Jiang

Cao

Butterworth

et al. 2023

IEEE/ASME Trans. Mechatron.

View full text Add to dashboard Cite

Picking up transparent objects is still a challenging task for robots. The visual properties of transparent objects such as reflection and refraction make the current grasping methods that rely on camera sensing fail to detect and localise them. However, humans can handle the transparent object well by first observing its coarse profile and then poking an area of interest to get a fine profile for grasping. Inspired by this, we propose a novel framework of vision-guided tactile poking for transparent objects grasping. In the proposed framework, a segmentation network is first used to predict the horizontal upper regions named as poking regions, where the robot can poke the object to obtain a good tactile reading while leading to minimal disturbance to the object's state. A poke is then performed with a high-resolution GelSight tactile sensor. Given the local profiles improved with the tactile reading, a heuristic grasp is planned for grasping the transparent object. To mitigate the limitations of real-world data collection and labelling for transparent objects, a large-scale realistic synthetic dataset was constructed. Extensive experiments demonstrate that our proposed segmentation network can predict the potential poking region with a high mean Average Precision (mAP) of 0.360, and the vision-guided tactile poking can enhance the grasping success rate significantly from 38.9% to 85.2%. Thanks to its simplicity, our proposed approach could also be adopted by other force or tactile sensors and could be used for grasping of other challenging objects. All the materials used in this paper are available at https://sites.google.com/view/tactilepoking.

show abstract

“…With the proposed combined sensor, they reconstruct the surface of an unknown object with a low error. Yang and Lepora 74 presented an exploration approach combined with passive vision and active touch. Park et al designed a large-area electrical impedance tomography-based tactile sensor, which can be used for contact contour detection.…”

Section: Planning-level Perceptionmentioning

confidence: 99%

A review on sensory perception for dexterous robotic manipulation

Xia¹,

Deng

Fang

et al. 2022

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

Sensory perception for dexterous robotic hands is an active research area and recent progress in robotics. Effective dexterous manipulation requires robotic hands to accurately feedback their state or perceive the surrounding environment. This article reviews the state-of-the-art of sensory perception for dexterous robotic manipulation. Two types of sensors, such as intrinsic and extrinsic sensors, are introduced according to their function and layout in robotic hands. These sensors provide rich information to a robotic hand, which contains the posture, the contact information of objects, and the physical information of the environment. Then, a comprehensive analysis of perception methods including planning-level, control-level, and learning-level perceptions is presented. The information obtained from sensory perception can help robotic hands to make decisions effectively. Previously issued reviews mainly focus on the design of tactile senor, while we analyze and discuss the relationship among sensing, perception, and dexterous manipulation. Some potential research topics on sensory perception are also summarized and discussed.

show abstract

Object exploration using vision and active touch

Cited by 10 publications

References 27 publications

Pressure distribution classification and segmentation of human hands in contact with the robot body

Pressure distribution classification and segmentation of human hands in contact with the robot body

Where Shall I Touch? Vision-Guided Tactile Poking for Transparent Object Grasping

A review on sensory perception for dexterous robotic manipulation

Contact Info

Product

Resources

About