Deep Learning for Tactile Understanding From Visual and Haptic Data

Gao, Yang; Hendricks, Lisa Anne; Kuchenbecker, Katherine J.; Darrell, Trevor

doi:10.48550/arxiv.1511.06065

Cited by 6 publications

(8 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multi-modal perception techniques are usually adopted to improve accuracy of the object classification algorithm by exploiting both visual and tactile data in the training phase. The deep learning method based on Convolutional Neural Networks (CNNs) proposed in [17] achieves very good performance in recognizing some material properties. The algorithm presented in [18] fuses visual and range data to recognize objects, while [19] combines visual features with tactile glances to refine object models, obtaining more accurate information about surfaces.…”

Section: Related Workmentioning

confidence: 99%

A Transfer Learning Approach to Cross-Modal Object Recognition: From Visual Observation to Robotic Haptic Exploration

Falco

Natale

et al. 2019

IEEE Trans. Robot.

View full text Add to dashboard Cite

In this work, we introduce the problem of crossmodal visuo-tactile object recognition with robotic active exploration. With this term, we mean that the robot observes a set of objects with visual perception and, later on, it is able to recognize such objects only with tactile exploration, without having touched any object before. Using a machine learning terminology, in our application we have a visual training set and a tactile test set, or vice versa. To tackle this problem, we propose an approach constituted by four steps: finding a visuotactile common representation, defining a suitable set of features, transferring the features across the domains, and classifying the objects. We show the results of our approach using a set of 15 objects, collecting 40 visual examples and five tactile examples for each object. The proposed approach achieves an accuracy of 94.7%, which is comparable with the accuracy of the monomodal case, i.e., when using visual data both as training set and test set. Moreover, it performs well compared to the human ability, which we have roughly estimated carrying out an experiment with ten participants.

show abstract

Section: Related Workmentioning

confidence: 99%

A Transfer Learning Approach to Cross-Modal Object Recognition: From Visual Observation to Robotic Haptic Exploration

Falco

Natale

et al. 2019

IEEE Trans. Robot.

View full text Add to dashboard Cite

show abstract

“…Recently, researchers have proposed several learningbased methods that estimate tactile properties from touch sensors. Gao et al [24], for example, used a neural network to infer haptic adjectives (i.e., qualitative properties such as "bumpy" or "squishy") from a Biotac sensor. While their technical approach also uses a recurrent neural network, specifically a 1D convolutional network and recurrent network, the output of the sensor-an assortment of 32 timevarying physical measurements, such as fluid pressure and temperature -is substantially more limited.…”

Section: Learning For Tactile Sensingmentioning

confidence: 99%

Shape-independent hardness estimation using deep learning and a GelSight tactile sensor

Yuan

Zhu

Owens³

et al. 2017

2017 IEEE International Conference on Robotics and Automation (ICRA)

164

View full text Add to dashboard Cite

Abstract-Hardness is among the most important attributes of an object that humans learn about through touch. However, approaches for robots to estimate hardness are limited, due to the lack of information provided by current tactile sensors. In this work, we address these limitations by introducing a novel method for hardness estimation, based on the GelSight tactile sensor, and the method does not require accurate control of contact conditions or the shape of objects. A GelSight has a soft contact interface, and provides high resolution tactile images of contact geometry, as well as contact force and slip conditions. In this paper, we try to use the sensor to measure hardness of objects with multiple shapes, under a loosely controlled contact condition. The contact is made manually or by a robot hand, while the force and trajectory are unknown and uneven. We analyze the data using a deep constitutional (and recurrent) neural network. Experiments show that the neural net model can estimate the hardness of objects with different shapes and hardness ranging from 8 to 87 in Shore 00 scale.

show abstract

“…In [18] the performance of CNNs is compared with that of LSTM for the classification of Visual and Haptic Data in a robotics setting, and in [28] the signals produced by wearable sensors are transformed into images so that Deep CNNs can be used for classification.…”

Section: Literature Reviewmentioning

confidence: 99%

Deep Learning for Time-Series Analysis

Gamboa¹

2017

Preprint

View full text Add to dashboard Cite

In many real-world application, e.g., speech recognition or sleep stage classification, data are captured over the course of time, constituting a Time-Series. Time-Series often contain temporal dependencies that cause two otherwise identical points of time to belong to different classes or predict different behavior. This characteristic generally increases the difficulty of analysing them. Existing techniques often depended on hand-crafted features that were expensive to create and required expert knowledge of the field. With the advent of Deep Learning new models of unsupervised learning of features for Time-series analysis and forecast have been developed. Such new developments are the topic of this paper: a review of the main Deep Learning techniques is presented, and some applications on Time-Series analysis are summaried. The results make it clear that Deep Learning has a lot to contribute to the field.

show abstract

Deep Learning for Tactile Understanding From Visual and Haptic Data

Cited by 6 publications

References 7 publications

A Transfer Learning Approach to Cross-Modal Object Recognition: From Visual Observation to Robotic Haptic Exploration

A Transfer Learning Approach to Cross-Modal Object Recognition: From Visual Observation to Robotic Haptic Exploration

Shape-independent hardness estimation using deep learning and a GelSight tactile sensor

Deep Learning for Time-Series Analysis

Contact Info

Product

Resources

About