Sensing the Frictional State of a Robotic Skin via Subtractive Color Mixing

Xi, Lin; Wiertlewski, Michaël

doi:10.1109/lra.2019.2893434

Cited by 58 publications

(34 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The surface pattern of the contact object is seen through the transparent elastomer, and force is evaluated based on the shape and displacement of the image of the eight conical feet, captured through a light conductive plate. In [92,93], multiple color filters and multiple LEDs with different colors were used, respectively, to obtain the deformation of sensor skin as color changes. In [92], a subtractive color mixing process using yellow and magenta translucent markers placed at different depths in the transparent elastomer body was used to estimate the three-dimensional displacement field.…”

Section: Physical Contact To Light Conversionmentioning

confidence: 99%

“…In [92,93], multiple color filters and multiple LEDs with different colors were used, respectively, to obtain the deformation of sensor skin as color changes. In [92], a subtractive color mixing process using yellow and magenta translucent markers placed at different depths in the transparent elastomer body was used to estimate the three-dimensional displacement field. If tactile information, such as force, is encoded as intensity or color information in each pixel value, the computational cost can be low because we can just see each pixel value to know the tactile information.…”

Section: Physical Contact To Light Conversionmentioning

confidence: 99%

See 1 more Smart Citation

Tactile Image Sensors Employing Camera: A Review

Shimonomura

2019

Sensors

134

View full text Add to dashboard Cite

A tactile image sensor employing a camera is capable of obtaining rich tactile information through image sequences with high spatial resolution. There have been many studies on the tactile image sensors from more than 30 years ago, and, recently, they have been applied in the field of robotics. Tactile image sensors can be classified into three typical categories according to the method of conversion from physical contact to light signals: Light conductive plate-based, marker displacement- based, and reflective membrane-based sensors. Other important elements of the sensor, such as the optical system, image sensor, and post-image analysis algorithm, have been developed. In this work, the literature is surveyed, and an overview of tactile image sensors employing a camera is provided with a focus on the sensing principle, typical design, and variation in the sensor configuration.

show abstract

Section: Physical Contact To Light Conversionmentioning

confidence: 99%

Section: Physical Contact To Light Conversionmentioning

confidence: 99%

Tactile Image Sensors Employing Camera: A Review

Shimonomura

2019

Sensors

134

View full text Add to dashboard Cite

show abstract

“…This advantage is fully exploited by GelSight-like sensors that use the captured raw image, for instance, to reconstruct the contacted surface geometry. In opposition, marker-based optical tactile sensors only track the displacement of sparse markers/pins printed in the soft deformable membrane [18], [19], [20], [21]. Consequently, the generation of synthetic tactile images depends on the corresponding tactile sensor working principle, especially in simulating the soft membrane physical properties.…”

Section: Related Work a Simulation Of Optical Tactile Sensorsmentioning

confidence: 99%

Generation of GelSight Tactile Images for Sim2Real Learning

Gomes

Paoletti

Luo

2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Most current works in Sim2Real learning for robotic manipulation tasks leverage camera vision that may be significantly occluded by robot hands during the manipulation. Tactile sensing offers complementary information to vision and can compensate for the information loss caused by the occlusions. However, the use of tactile sensing is restricted in the Sim2Real research due to no simulated tactile sensors being available. To mitigate the gap, we introduce a novel approach for simulating a GelSight tactile sensor in the commonly used Gazebo simulator. Similar to the real GelSight sensor, the simulated sensor can produce high-resolution images from depth-maps captured by a simulated optical sensor, and reconstruct the interaction between the touched object and an opaque soft membrane. It can indirectly sense forces, geometry, texture and other properties of the object and enables Sim2Real learning with tactile sensing. Preliminary experimental results have shown that the simulated sensor could generate realistic outputs similar to the ones captured by a real GelSight sensor. All the materials used in this paper are available at https://danfergo.github.io/gelsightsimulation.

show abstract

“…Like in the case of the GelForce sensor, force and pressure applied to the sensor are inferred from the movements of these fluorescent green markers which are tracked by computer vision algorithm. Moreover, Lin and Wiertlewski [36] developed a visuotactile device in 2019 with embedded dye markers in the flexible material. Instead of spherical makers, semi-transparent two color dye markers arrange in two layer arrays that overlap as shown in Fig.…”

Section: Visuotactile Sensorsmentioning

confidence: 99%

Visuotactile Sensors With Emphasis on GelSight Sensor: A Review

2020

View full text Add to dashboard Cite

This review paper focuses on vision and touch-based sensors known as visuotactile. The study of visuotactile sensation and perception became a multidisciplinary field of study by philosophers, psychologists, biologists, engineers, technologists, and roboticists in the fields of haptics, machine vision, and artificial intelligence and it dates back centuries. To the best of our knowledge, the earliest records of visuotactile sensor was not applied to robotics and was not even for hand or finger imprint analysis yet for recording the foot pressure distribution of a walking or standing human known as pedobarograph. Our review paper presents the different literature related to visuotactile sensors that lead to a high-resolution miniature pedobarographlike sensor known as the GelSight sensor. Moreover, this review paper focuses on architecture, different techniques, hardware, and software development of GelSight sensor since 2009 with its applications in haptics, robotics, and computer vision.

show abstract

Sensing the Frictional State of a Robotic Skin via Subtractive Color Mixing

Cited by 58 publications

References 28 publications

Tactile Image Sensors Employing Camera: A Review

Tactile Image Sensors Employing Camera: A Review

Generation of GelSight Tactile Images for Sim2Real Learning

Visuotactile Sensors With Emphasis on GelSight Sensor: A Review

Contact Info

Product

Resources

About