Towards vision-based robotic skins: a data-driven, multi-camera tactile sensor

Trueeb, Camill; Sferrazza, Carmelo; D’Andrea, Raffaello

doi:10.1109/robosoft48309.2020.9116060

Cited by 43 publications

(19 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In an extension study, the field of view was increased by placing four cameras next to each other (Trueeb et al, 2020). The cameras were equipped with close-focus lenses, which helped in reducing the overall thickness down to 17.45 mm.…”

Section: Imaging-based Tactile Sensingmentioning

confidence: 99%

Tactile Sensing for Minimally Invasive Surgery: Conventional Methods and Potential Emerging Tactile Technologies

Othman¹,

Lai²,

Abril³

et al. 2022

Front. Robot. AI

View full text Add to dashboard Cite

As opposed to open surgery procedures, minimally invasive surgery (MIS) utilizes small skin incisions to insert a camera and surgical instruments. MIS has numerous advantages such as reduced postoperative pain, shorter hospital stay, faster recovery time, and reduced learning curve for surgical trainees. MIS comprises surgical approaches, including laparoscopic surgery, endoscopic surgery, and robotic-assisted surgery. Despite the advantages that MIS provides to patients and surgeons, it remains limited by the lost sense of touch due to the indirect contact with tissues under operation, especially in robotic-assisted surgery. Surgeons, without haptic feedback, could unintentionally apply excessive forces that may cause tissue damage. Therefore, incorporating tactile sensation into MIS tools has become an interesting research topic. Designing, fabricating, and integrating force sensors onto different locations on the surgical tools are currently under development by several companies and research groups. In this context, electrical force sensing modality, including piezoelectric, resistive, and capacitive sensors, is the most conventionally considered approach to measure the grasping force, manipulation force, torque, and tissue compliance. For instance, piezoelectric sensors exhibit high sensitivity and accuracy, but the drawbacks of thermal sensitivity and the inability to detect static loads constrain their adoption in MIS tools. Optical-based tactile sensing is another conventional approach that facilitates electrically passive force sensing compatible with magnetic resonance imaging. Estimations of applied loadings are calculated from the induced changes in the intensity, wavelength, or phase of light transmitted through optical fibers. Nonetheless, new emerging technologies are also evoking a high potential of contributions to the field of smart surgical tools. The recent development of flexible, highly sensitive tactile microfluidic-based sensors has become an emerging field in tactile sensing, which contributed to wearable electronics and smart-skin applications. Another emerging technology is imaging-based tactile sensing that achieved superior multi-axial force measurements by implementing image sensors with high pixel densities and frame rates to track visual changes on a sensing surface. This article aims to review the literature on MIS tactile sensing technologies in terms of working principles, design requirements, and specifications. Moreover, this work highlights and discusses the promising potential of a few emerging technologies towards establishing low-cost, high-performance MIS force sensing.

show abstract

Section: Imaging-based Tactile Sensingmentioning

confidence: 99%

Tactile Sensing for Minimally Invasive Surgery: Conventional Methods and Potential Emerging Tactile Technologies

Othman¹,

Lai²,

Abril³

et al. 2022

Front. Robot. AI

View full text Add to dashboard Cite

show abstract

“…There are various studies [49] on force calibration of vision-based tactile sensors. Included are using convolutional neural networks (CNNs), that are useful when dealing with images [31,50]. However, in our case, there are two sensing modalities (force and tactile) presented in each image, hence we cannot use the exact same method as above.…”

Section: Sensor Calibration and Validationmentioning

confidence: 99%

F-TOUCH Sensor: Concurrent Geometry Perception and Multi-Axis Force Measurement

Alomainy

Vitanov

et al. 2021

IEEE Sensors J.

View full text Add to dashboard Cite

Force and tactile sensing has experienced a surge of interest over recent decades, as it conveys a range of information through physical interaction. Tactile sensors aim to obtain tactile information (including pressure, texture etc.). However, current tactile sensors have difficulties in accurately acquiring force signals with regards to magnitude and direction. This is because tactile sensors such as the GelSight sensor estimate shear forces from discrete markers embedded in a compliant sensor interface, employing image processing techniquesthe resultant force errors are sizeable. This paper presents a novel design for a force/tactile sensor, namely the F-TOUCH (Force and Tactile Optically Unified Coherent Haptics) sensor, representing an advancement on current vision-based tactile sensors. In addition to acquiring geometric features at a high spatial resolution, our sensor incorporates a number of deformable structural elements allowing us to measure translational and rotational force and torque along six axes with high accuracy. The proposed sensor contains three key components: a coated elastomer layer acting as the compliant sensing medium, spring mechanisms acting as deformable structural elements, and a camera for image capture. The camera records the deformation of the structural elements as well as the distortion of the compliant sensing medium, concurrently acquiring force and tactile information. The sensor is calibrated with the use of a commercial ATI force sensor. An experimental study shows that the F-TOUCH sensor outperforms the GelSight sensor with regard to its capabilities to sense force signals and capturing the geometry of the contacted object. Index Terms-Tactile sensing; force/torque sensor; contact sensing; vision; image processing; robotics I. INTRODUCTION orce and tactile sensing has garnered much research interest over past decades, for it admits of information gathering through direct physical contact between a sensing device and

show abstract

“…In [28], green florescent particles are randomly distributed within the soft elastomer with black opaque coating so that a higher number of markers can be tracked and used to predict the interaction with the object, according to the authors. In [29], a sensor with the same membrane construction method, 4 Raspberry PI cameras and fisheye lenses has been proposed for optical tactile skins.…”

Section: Marker-based Optical Tactile Sensorsmentioning

confidence: 99%

GelTip Tactile Sensor for Dexterous Manipulation in Clutter

Gomes¹,

Si²

2021

Preprint

View full text Add to dashboard Cite

Tactile sensing is an essential capability for robots that carry out dexterous manipulation tasks. While cameras, Lidars and other remote sensors can assess a scene globally and instantly, tactile sensors can reduce their measurement uncertainties and gain information about the local physical interactions between the in-contact objects and the robot, that are often not accessible via remote sensing. Tactile sensors can be grouped into two main categories: electronic tactile skins and camera based optical tactile sensors. The former are slim and can be fitted to different body parts, whereas the latter assume a more prismatic shape and have much higher sensing resolutions, offering a good advantage for being used as robotic fingers or fingertips. One of such optical tactile sensors is our GelTip sensor that is shaped as a finger and can sense contacts on any location of its surface. As such, the GelTip sensor is able to detect contacts from all the directions, like a human finger. To capture these contacts, it uses a camera installed at its base to track the deformations of the opaque elastomer that covers its hollow, rigid and transparent body. Thanks to this design, a gripper equipped with GelTip sensors is capable of simultaneously monitoring contacts happening inside and outside its grasp closure. Experiments carried out using this sensor demonstrate how contacts can be localised, and more importantly, the advantages, and even possibly a necessity, of leveraging all-around touch sensing in dexterous manipulation tasks in clutter where contacts may happen at any location of the finger. In particular, experiments carried out in a Blocks World environment show that the detected contacts on the fingers can be used to adapt planned actions during the different moments of the reach-to-grasp motion. All the materials for the fabrication of the GelTip sensor can be found at https://danfergo.github.io/geltip/

show abstract

Towards vision-based robotic skins: a data-driven, multi-camera tactile sensor

Cited by 43 publications

References 16 publications

Tactile Sensing for Minimally Invasive Surgery: Conventional Methods and Potential Emerging Tactile Technologies

Tactile Sensing for Minimally Invasive Surgery: Conventional Methods and Potential Emerging Tactile Technologies

F-TOUCH Sensor: Concurrent Geometry Perception and Multi-Axis Force Measurement

GelTip Tactile Sensor for Dexterous Manipulation in Clutter

Contact Info

Product

Resources

About