Robust material classification with a tactile skin using deep learning

Baishya, Shiv Sankar; Bäuml, Berthold

doi:10.1109/iros.2016.7758088

Cited by 107 publications

(55 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Es decir, la secuencia temporal es una colección de datos táctiles obtenidos durante el cierre de la mano o pinza hasta envolver la superficie del objeto, registrando los valores de presión generados por el contacto hasta alcanzar un agarre considerado como estable (un agarre capaz de sujetar el objeto sin que este se deslice). Por lo general, las secuencias de datos táctiles de este tipo se emplean para el reconocimiento del objeto en base a características de rigidez y flexibilidad del material con el que está fabricado, como hicieron Baishya and Bäuml (2016), o para controlar la deformación, como demuestran Delgado et al (2017). No obstante, durante la acción de cierre de la mano y aun cuando la mano no cambie su configuración, es posible que el objeto cambie su posición dentro de la mano.…”

Section: Percepción Táctilunclassified

Clasificación de objetos usando percepción bimodal de palpación única en acciones de agarre robótico

Velasco

Zapata-Impata

Gil

et al. 2020

Rev. iberoam. autom. inform. ind.

View full text Add to dashboard Cite

<p>Este trabajo presenta un método para clasificar objetos agarrados con una mano robótica multidedo combinando en un descriptor híbrido datos propioceptivos y táctiles. Los datos propioceptivos se obtienen a partir de las posiciones articulares de la mano y los táctiles se extraen del contacto registrado por células de presión instaladas en las falanges. La aproximación propuesta permite identificar el objeto aprendiendo de forma implícita su geometría y rigidez usando los datos que facilitan los sensores. En este trabajo demostramos que el uso de datos bimodales con técnicas de aprendizaje supervisado mejora la tasa de reconocimiento. En la experimentación, se han llevado a cabo más de 3000 agarres de hasta 7 objetos domésticos distintos, obteniendo clasificaciones correctas del 95%con métrica F1, realizando una única palpación del objeto. Además, la generalización del método se ha verificado entrenando nuestro sistema con unos objetos y posteriormente, clasificando otros nuevos similares.</p>

show abstract

Section: Percepción Táctilunclassified

Clasificación de objetos usando percepción bimodal de palpación única en acciones de agarre robótico

Velasco

Zapata-Impata

Gil

et al. 2020

Rev. iberoam. autom. inform. ind.

View full text Add to dashboard Cite

show abstract

“…Soon afterwards, deep learning methods also found their way to applications where tactile sensing is involved [21]- [24]. Many of these studies, however, deal with the classification problem in order to e.g., recognize objects inside a robotic hand [21], recognize materials [22], [23] and properties [24] from touch and image. Yuan et al [25] estimated object hardness as a continuous value using tactile sensor through supervised learning.…”

Section: B Recognition Through Tactile Sensingmentioning

confidence: 99%

Deep Visuo-Tactile Learning: Estimation of Tactile Properties from Images

Takahashi

Tan

2019

2019 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Estimation of tactile properties from vision, such as slipperiness or roughness, is important to effectively interact with the environment. These tactile properties help us decide which actions we should choose and how to perform them. E.g., we can drive slower if we see that we have bad traction or grasp tighter if an item looks slippery. We believe that this ability also helps robots to enhance their understanding of the environment, and thus enables them to tailor their actions to the situation at hand. We therefore propose a model to estimate the degree of tactile properties from visual perception alone (e.g., the level of slipperiness or roughness). Our method extends a encoderdecoder network, in which the latent variables are visual and tactile features. In contrast to previous works, our method does not require manual labeling, but only RGB images and the corresponding tactile sensor data. All our data is collected with a webcam and uSkin tactile sensor mounted on the end-effector of a Sawyer robot, which strokes the surfaces of 25 different materials. 1 We show that our model generalizes to materials

show abstract

“…However, the most important difference is that we do not use manually designed feature descriptors but use a deep learning based end-to-end learning scheme with automatic feature extraction directly from the 16000 dimensional raw input signal. End-to-end learning is important not only because it makes the time-consuming feature design by an expert obsolete but because the automatic feature extraction gives better performance for the flexible skin sensor as was shown in [2] (different feature descriptors and classification methods including SVMs have been compared to deep learning).…”

Section: A Related Workmentioning

confidence: 99%

“…As the physical interactions between the sensor and surface while sliding over it are involved, usually learning has to be used to interpret the tactile data. Deep learning has shown very robust [2] and even superhuman performance [1] in tactile material classification. But learning typically needs a large number of training samples and in robotics, sample acquisition is costly, even more so for active sensing tasks like tactile sensing.…”

Section: Introductionmentioning

confidence: 99%

Deep n-Shot Transfer Learning for Tactile Material Classification with a Flexible Pressure-Sensitive Skin

Bäuml¹,

Tulbure²

2019

2019 International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

n-shot learning, i.e., learning a classifier from only few or even one training samples per class, is the ultimate goal in minimizing the cost of sample acquisition. This is esp. important for active sensing tasks like tactile material classification. Achieving high classification accuracy from only few samples is typically possible only when pre-knowledge is used. In n-shot transfer learning, knowledge from pre-training on a large knowledge set with many classes and samples per class has to be transferred to support the training for a given task set with only few samples per new class.In this paper, we show for the first time that deep end-to-end transfer learning is feasible for tactile material classification. Based on the previously presented (TactNet-II) [1], a deep convolutional neural network (CNN) which reaches superhuman tactile classification performance, we adapt state-of-the art deep transfer learning methods.We evaluate the resulting deep n-shot learning methods with a publicly available tactile material data set with 36 materials [1] in a 6-way n-shot learning task with 30 materials in the knowledge set. In 1-shot learning, our deep transfer learning method reaches 75.5% classification accuracy and in 10-shot more than 90%, outperforming classification without knowledge transfer by more than 40%. This results in an up to 15 time reduction in the number of samples needed to reach a desired accuracy level.We also provide insights of the inner workings of the derived deep transfer learning methods.

show abstract

Robust material classification with a tactile skin using deep learning

Cited by 107 publications

References 20 publications

Clasificación de objetos usando percepción bimodal de palpación única en acciones de agarre robótico

Clasificación de objetos usando percepción bimodal de palpación única en acciones de agarre robótico

Deep Visuo-Tactile Learning: Estimation of Tactile Properties from Images

Deep n-Shot Transfer Learning for Tactile Material Classification with a Flexible Pressure-Sensitive Skin

Contact Info

Product

Resources

About