Tactile Mapping and Localization from High-Resolution Tactile Imprints

Bauzá, Maria; Canal, Oleguer; Rodríguez, Alberto

doi:10.1109/icra.2019.8794298

Cited by 85 publications

(71 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…When the object makes contact with the sensor surface, the elastomer surface deforms, and a shading image reflecting these deformations appears in the image obtained from the camera. This method has been used in GelSight, which is currently the most popular tactile image sensor [32,36,38,40,42,76,77,78,79,80], and some other sensors [37,41,43].…”

Section: Physical Contact To Light Conversionmentioning

confidence: 99%

“…For the reflective membrane method, the camera image includes a fine surface texture and clear outline shape of the contact object. In [37,38,80], in-hand object localization was performed through pattern matching. A typical reflective membrane-based sensor, GelSight, was applied for many tactile sensing tasks, such as recognizing surface textures [36].…”

Section: Image Analysis For Extracting Tactile Informationmentioning

confidence: 99%

See 1 more Smart Citation

Tactile Image Sensors Employing Camera: A Review

Shimonomura

2019

Sensors

133

View full text Add to dashboard Cite

A tactile image sensor employing a camera is capable of obtaining rich tactile information through image sequences with high spatial resolution. There have been many studies on the tactile image sensors from more than 30 years ago, and, recently, they have been applied in the field of robotics. Tactile image sensors can be classified into three typical categories according to the method of conversion from physical contact to light signals: Light conductive plate-based, marker displacement- based, and reflective membrane-based sensors. Other important elements of the sensor, such as the optical system, image sensor, and post-image analysis algorithm, have been developed. In this work, the literature is surveyed, and an overview of tactile image sensors employing a camera is provided with a focus on the sensing principle, typical design, and variation in the sensor configuration.

show abstract

Section: Physical Contact To Light Conversionmentioning

confidence: 99%

Section: Image Analysis For Extracting Tactile Informationmentioning

confidence: 99%

Tactile Image Sensors Employing Camera: A Review

Shimonomura

2019

Sensors

133

View full text Add to dashboard Cite

show abstract

“…We relied on black-white 2D images on coin recognition. Therefore, the authors argued that object recognition in GelSight can be done without 3D image reconstruction in previous studies [6], [16], [17]. This simple mechanical filter without any electronics, objects recognition can be achieved.…”

Section: ) Prototypementioning

confidence: 99%

“…Current GelSight sensors can be divided into two categories: with and without permanent grid dots or markers. GelSight sensor used in the measurement of surface texture [1] and microgeometry [9], lump detection [2], and tactile mapping and localization [16] do not use grid dots while GelSight sensor used in the measurement of shear and slip [6], [7], [17]- [19] used the permanent markers in the elastomer. The use of permanent markers in the GelSight sensor was first introduced by Yuan et al [10].…”

Section: Introductionmentioning

confidence: 99%

Low-cost GelSight with UV Markings: Feature Extraction of Objects Using AlexNet and Optical Flow without 3D Image Reconstruction

Abad

Ranasinghe

2020

2020 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

GelSight sensor has been used to study microgeometry of objects since 2009 in tactile sensing applications. Elastomer, reflective coating, lighting, and camera were the main challenges of making a GelSight sensor within a short period. The recent addition of permanent markers to the GelSight was a new era in shear/slip studies. In our previous studies, we introduced Ultraviolet (UV) ink and UV LEDs as a new form of marker and lighting respectively. UV ink markers are invisible using ordinary LED but can be made visible using UV LED. Currently, recognition of objects or surface textures using GelSight sensor is done using fusion of camera-only images and GelSight captured images with permanent markings. Those images are fed to Convolutional Neural Networks (CNN) to classify objects. However, our novel approach in using low-cost GelSight sensor with UV markings, the 3D height map to 2D image conversion, and the additional non-Gelsight captured images for training the CNN can be eliminated. AlexNet and optical flow algorithm have been used for feature recognition of five coins without UV markings and shear/slip of the coin in GelSight with UV markings respectively. Our results on confusion matrix show that, on average coin recognition can reach 93.4% without UV markings using AlexNet. Therefore, our novel method of using GelSight with UV markings would be useful to recognize full/partial object, shear/slip, and force applied to the objects without any 3D image reconstruction.

show abstract

“…Among different technologies for tactile sensing, vision-based tactile sensors, such as GelSight (which uses a camera to track markers on a deformable silicone gel pad) [2] or GelSlim [3], are promising due to their high-resolution, convenient data-multiplexing and embedded compliance. However, research on vision-based tactile sensors [4], [5], [6], [7], [8] has focused more intensely on measuring geometry instead of on force reconstruction.…”

Section: Introductionmentioning

confidence: 99%

Dense Tactile Force Estimation using GelSlim and inverse FEM

Donlon

Dong

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

Self Cite

114

View full text Add to dashboard Cite

A sequence of Kendama manipulations with corresponding displacement field (yellow) and force field (red). Video can be found on Youtube: https://youtu.be/hWw9A0ZBZuUAbstract-In this paper, we present a new version of tactile sensor GelSlim 2.0 with the capability to estimate the contact force distribution in real time. The sensor is vision-based and uses an array of markers to track deformations on a gel pad due to contact. A new hardware design makes the sensor more rugged, parametrically adjustable and improves illumination. Leveraging the sensor's increased functionality, we propose to use inverse Finite Element Method (iFEM), a numerical method to reconstruct the contact force distribution based on marker displacements. The sensor is able to provide force distribution of contact with high spatial density. Experiments and comparison with ground truth show that the reconstructed force distribution is physically reasonable with good accuracy.

show abstract

Tactile Mapping and Localization from High-Resolution Tactile Imprints

Cited by 85 publications

References 42 publications

Tactile Image Sensors Employing Camera: A Review

Tactile Image Sensors Employing Camera: A Review

Low-cost GelSight with UV Markings: Feature Extraction of Objects Using AlexNet and Optical Flow without 3D Image Reconstruction

Dense Tactile Force Estimation using GelSlim and inverse FEM

Contact Info

Product

Resources

About