Estimating object hardness with a GelSight touch sensor

Yuan, Wenzhen; Srinivasan, Mandayam A.; Adelson, Edward H.

doi:10.1109/iros.2016.7759057

Cited by 73 publications

(43 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For example, contact geometry in pixel space could be used in combination with knowledge of grasping force and gel material properties to infer 3D local object geometry. If markers are placed on the gel surface, marker flow can be used to estimate object hardness [20] or shear forces [21]. These quantities, as well as the sensor's calibrated image output, can be used directly in model-based or learningbased approaches to robot grasping and manipulation.…”

Section: Discussionmentioning

confidence: 99%

GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger

Donlon

Dong

Liu

et al. 2018

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

265

171

View full text Add to dashboard Cite

This work describes the development of a highresolution tactile-sensing finger for robot grasping. This finger, inspired by previous GelSight sensing techniques, features an integration that is slimmer, more robust, and with more homogeneous output than previous vision-based tactile sensors.To achieve a compact integration, we redesign the optical path from illumination source to camera by combining light guides and an arrangement of mirror reflections. We parameterize the optical path with geometric design variables and describe the tradeoffs between the finger thickness, the depth of field of the camera, and the size of the tactile sensing area.The sensor sustains the wear from continuous use -and abuse -in grasping tasks by combining tougher materials for the compliant soft gel, a textured fabric skin, a structurally rigid body, and a calibration process that maintains homogeneous illumination and contrast of the tactile images during use. Finally, we evaluate the sensor's durability along four metrics that track the signal quality during more than 3000 grasping experiments.

show abstract

Section: Discussionmentioning

confidence: 99%

GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger

Donlon

Dong

Liu

et al. 2018

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

265

171

View full text Add to dashboard Cite

show abstract

“…In our previous work [5], we showed that GelSight can accurately estimate the hardness of a set of hemispherical silicone samples under loosely controlled contact conditions. As in this work, we asked a human tester and an open-loop robot to press on samples, and developed a model to predict the hardness according to both the changing brightness in the GelSight image and the displacement field of the surface markers.…”

Section: B Hardness Measurementmentioning

confidence: 99%

“…We take a sequence of images during the contact procedure as our input signal. showed in our previous work [5] that these physical changes could be measured using simple image cues: namely, changes in intensity and motion of the black markers embedded in the gel. Figure 2 shows an example.…”

Section: Introductionmentioning

confidence: 96%

Shape-independent hardness estimation using deep learning and a GelSight tactile sensor

Yuan

Zhu

Owens³

et al. 2017

2017 IEEE International Conference on Robotics and Automation (ICRA)

Self Cite

164

View full text Add to dashboard Cite

Abstract-Hardness is among the most important attributes of an object that humans learn about through touch. However, approaches for robots to estimate hardness are limited, due to the lack of information provided by current tactile sensors. In this work, we address these limitations by introducing a novel method for hardness estimation, based on the GelSight tactile sensor, and the method does not require accurate control of contact conditions or the shape of objects. A GelSight has a soft contact interface, and provides high resolution tactile images of contact geometry, as well as contact force and slip conditions. In this paper, we try to use the sensor to measure hardness of objects with multiple shapes, under a loosely controlled contact condition. The contact is made manually or by a robot hand, while the force and trajectory are unknown and uneven. We analyze the data using a deep constitutional (and recurrent) neural network. Experiments show that the neural net model can estimate the hardness of objects with different shapes and hardness ranging from 8 to 87 in Shore 00 scale.

show abstract

“…In [77] a robot leg equipped with an accelerometer is employed to actively knock on object surfaces and by analysing the sensor data, the hardness, elasticity and stiffness of the object can be revealed. In recent works [78], [79], the hardness of objects can also be estimated by processing the tactile image sequences from a GelSight sensor. In [80], multiple computational algorithms are applied to classify various materials based on mechanical impedances using tactile data and it is found that SVM performs best.…”

Section: B Object Stiffness Based Tactile Materials Recognitionmentioning

confidence: 99%

Robotic tactile perception of object properties: A review

et al. 2017

View full text Add to dashboard Cite

Abstract-Touch sensing can help robots understand their surrounding environment, and in particular the objects they interact with. To this end, roboticists have, in the last few decades, developed several tactile sensing solutions, extensively reported in the literature. Research into interpreting the conveyed tactile information has also started to attract increasing attention in recent years. However, a comprehensive study on this topic is yet to be reported. In an effort to collect and summarize the major scientific achievements in the area, this survey extensively reviews current trends in robot tactile perception of object properties. Available tactile sensing technologies are briefly presented before an extensive review on tactile recognition of object properties. The object properties that are targeted by this review are shape, surface material and object pose. The role of touch sensing in combination with other sensing sources is also discussed. In this review, open issues are identified and future directions for applying tactile sensing in different tasks are suggested.

show abstract

Estimating object hardness with a GelSight touch sensor

Cited by 73 publications

References 20 publications

GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger

GelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger

Shape-independent hardness estimation using deep learning and a GelSight tactile sensor

Robotic tactile perception of object properties: A review

Contact Info

Product

Resources

About