Texture recognition based on multi-sensory integration of proprioceptive and tactile signals

Abstract

The sense of touch plays a fundamental role in enabling us to interact with our surrounding environment. Indeed, the presence of tactile feedback in prostheses greatly assists amputees in doing daily tasks. In this line, the present study proposes an integration of artificial tactile and proprioception receptors for texture discrimination under varying scanning speeds. Here, we fabricated a soft biomimetic fingertip including an 8 × 8 array tactile sensor and a piezoelectric sensor to mimic Merkel, Meissner, and Pacinian mechanoreceptors in glabrous skin, respectively. A hydro-elastomer sensor was fabricated as an artificial proprioception sensor (muscle spindles) to assess the instantaneous speed of the biomimetic fingertip. In this study, we investigated the concept of the complex receptive field of RA-I and SA-I afferents for naturalistic textures. Next, to evaluate the synergy between the mechanoreceptors and muscle spindle afferents, ten naturalistic textures were manipulated by a soft biomimetic fingertip at six different speeds. The sensors’ outputs were converted into neuromorphic spike trains to mimic the firing pattern of biological mechanoreceptors. These spike responses are then analyzed using machine learning classifiers and neural coding paradigms to explore the multi-sensory integration in real experiments. This synergy between muscle spindle and mechanoreceptors in the proposed neuromorphic system represents a generalized texture discrimination scheme and interestingly irrespective of the scanning speed.

Neuromorphic Tactile Sensing System for Textural Features Classification

“…The motivation of this choice is to consider mechanoreceptors associated with texture perception with a minimal computational cost. The Ruffini ending (SA-II) is excluded as it is not implicated in texture perception [35], [43], [44]. RA-II was omitted due to its higher computational cost in modeling when compared to SA-I and RA-I receptors (refer to supplementary material Section 1.2 for details).…”

Simultaneous Velocity and Texture Classification from a Neuromorphic Tactile Sensor Using Spiking Neural Networks

“…Whereas many works seek to achieve texture classification regardless of exploratory movement, the classification of specific movements is an area not often explored within artificial tactile sensing literature. Kinesthesis within robotic systems is often performed by processing the output of a purpose built sensor [28,29]. While perhaps analogous to classifying known movements, specific movements must be detected amid noisy signals in the field of tactile slip detection.…”

Porous Nanocomposites with Enhanced Intrinsic Piezoresistive Sensitivity for a Highly Integrated Multimodal Tactile Sensor

“…The third type measures multidimensional stress fields by using high-density array pressure sensors, which are commonly based on micro/nanofabrication technologies, presenting complex structures and poor reliability. 43,44 In our study, we proposed a novel sensor utilizing highly porous materials with an outstanding intrinsic resistance sensitivity, enabling continuous measurement of tangential forces theoretically. Furthermore, the sensor's fabrication process is straightforward.…”