Minsight: A Fingertip‐Sized Vision‐Based Tactile Sensor for Robotic Manipulation

Andrussow, Iris; Sun, Huanbo; Kuchenbecker, Katherine J.; Martius, Georg

doi:10.1002/aisy.202300042

Cited by 12 publications

(1 citation statement)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In previous studies, tactile sensors based on different principles have been proposed, mainly including optical (infrared or visible light) [58] , piezoresistive [59] , piezoelectric [60] , capacitive [61] , strain gauges [62] and accelerator [63] . Among them, capacitive tactile sensor arrays have higher sensitivity and dynamic characteristics, while piezoresistive tactile sensors are inexpensive.…”

Section: Introduction Of Tactile Sensorsmentioning

confidence: 99%

Promising real-time fruit and vegetable quality detection technologies applicable to manipulator picking process

Jintao,

Qinyi,

Hao

et al. 2024

International Journal of Agricultural and Biological Engineering

View full text Add to dashboard Cite

In recent years, worldwide research on fruit and vegetable quality detection technology includes machine vision, spectroscopy, acoustic vibration, tactile sensors, etc. These technologies have also been gradually applied to fruit and vegetable grading and sorting lines in recent years, greatly improving the income of farmers. There have been numerous reviews of these techniques. Most of the published research on fruit and vegetable quality detection technology is still carried out in the laboratory. The emphases have been on quality feature extraction, model establishment and experimental verification. The successful application in the fruit and vegetable sorting production line proves that these studies have high application potential and value, and we look forward to the performance of these sensing technologies in the fruit and vegetable picking field. Therefore, in this paper, based on the future highly automated fruit and vegetable picking mode, we will focus on three kinds of fruit and vegetable quality detection technologies including machine vision, tactile sensor and spectroscopy, to provide some reference for future research. Since there are currently limited cases of detecting quality during the fruit and vegetable picking, experiments performed on prototypes of manipulator, or devices such as Nanocilia sensors, portable spectrometers, etc., which are compact and convenient to mount on manipulator will be reviewed. Several tables and mosaics showing the performance of the three technologies in the detection of fruit and vegetable quality over the past five years have been listed. The performance of each sensing technology was relatively satisfactory in the laboratory in general. However, in the picking scenario, there are still many challenges to be solved. Different from industrial environments, agricultural scenarios are complex and changeable. Fragile and vulnerable agricultural products pose another challenge. The development of portable devices and nanomaterials have become important breakthroughs. Optical and tactile detection methods, as well as the integration of different quality detection methods, are expected to be the trends of research and development.

show abstract

Section: Introduction Of Tactile Sensorsmentioning

confidence: 99%

Promising real-time fruit and vegetable quality detection technologies applicable to manipulator picking process

Jintao,

Qinyi,

Hao

et al. 2024

International Journal of Agricultural and Biological Engineering

View full text Add to dashboard Cite

show abstract

Reconstructing Soft Robotic Touch via In‐Finger Vision

Guo,

Han,

Zhong

et al. 2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

Incorporating authentic tactile interactions into virtual environments presents a notable challenge for the emerging development of soft robotic metamaterials. In this study, a vision‐based approach is introduced to learning proprioceptive interactions by simultaneously reconstructing the shape and touch of a soft robotic metamaterial (SRM) during physical engagements. The SRM design is optimized to the size of a finger with enhanced adaptability in 3D interactions while incorporating a see‐through viewing field inside, which can be visually captured by a miniature camera underneath to provide a rich set of image features for touch digitization. Employing constrained geometric optimization, the proprioceptive process with aggregated multi‐handles is modeled. This approach facilitates real‐time, precise, and realistic estimations of the finger's mesh deformation within a virtual environment. Herein, a data‐driven learning model is also proposed to estimate touch positions, achieving reliable results with impressive R2 scores of 0.9681, 0.9415, and 0.9541 along the x, y, and z axes. Furthermore, the robust performance of the proposed methods in touch‐based human–cybernetic interfaces and human–robot collaborative grasping is demonstrated. In this study, the door is opened to future applications in touch‐based digital twin interactions through vision‐based soft proprioception.

show abstract

Disentangling Contact Location for Stretchable Tactile Sensors from Soft Waveguide Ultrasonic Scatter Signals

Li,

Lin,

Shull

et al. 2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

Flexible tactile sensors have the ability to provide unparalleled levels of tactile sensation, including information regarding roughness, contact force, and contact location. However, it remains a challenge to achieve precise contact location sensing that is decoupled from sensor strain and touching forces. This paper proposes a novel data‐driven approach for force contact location sensing (FCLS) with the influence of sensor strain and forces based on scatter signals (SS) of the ultrasonic waveguide. First, the envelope of the force contact scatter signal (FCSS) is extracted via the Hilbert transform, which retrieves the global features of SS. The time‐frequency spectrogram is obtained via continuous wavelet transform, which extracts the local features of SS. Second, a deep convolutional neural network (CNN) is utilized to extract these features separately and concentrate them together. Third, based on the outputs of the CNN, a multilayer perception regression model is applied to acquire the force contact location. The experimental results indicate that the accuracy of the proposed FCLS method has a mean absolute error of 0.627 mm and a mean relative error of 3.19%. This research provides a foundation for further multimodal sensing using ultrasonic waveguides and its application in robotic sensing.

show abstract

Minsight: A Fingertip‐Sized Vision‐Based Tactile Sensor for Robotic Manipulation

Cited by 12 publications

References 44 publications

Promising real-time fruit and vegetable quality detection technologies applicable to manipulator picking process

Promising real-time fruit and vegetable quality detection technologies applicable to manipulator picking process

Reconstructing Soft Robotic Touch via In‐Finger Vision

Disentangling Contact Location for Stretchable Tactile Sensors from Soft Waveguide Ultrasonic Scatter Signals

Contact Info

Product

Resources

About