Spatio-Temporal Encoding Improves Neuromorphic Tactile Texture Classification

Gupta, Anupam K.; Nakagawa-Silva, Andrei; Lepora, Nathan F.; Thakor, Nitish V.

doi:10.1109/jsen.2021.3087511

Cited by 12 publications

(6 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed solutions demand offline processing of raw signals, such as requiring a whole window of signal (time window) to compute handcrafted features, before fetching the decision or action, which correspondingly requires high computational resources such as memory footprint, power consumption, and high inference time. Despite the wide range of proposed features, varying from simple and basic features as in [13] and [15] to more complex features as in [14] and [17], the drawback persists, by requiring a time window of signal for processing. Furthermore, most of these works employed the Izhikevich model for encoding and spike conversion.…”

Section: B Related Workmentioning

confidence: 99%

Neuromorphic Tactile Sensing System for Textural Features Classification

Al Haj Ali,

Abbass,

Gianoglio

et al. 2024

IEEE Sensors J.

View full text Add to dashboard Cite

Artificial tactile sensing systems have gained significant attention in recent years due to their potential to enhance human-machine interaction. Numerous initiatives have been introduced to shift the computational paradigms of these systems towards a more biologically inspired approach, by incorporating neuromorphic computing methods. Despite the significant advances made by these systems, dependence on complex offline methods for classification (i.e. hand-crafted encoding features), remains a limitation for their real-time deployment. In this work, we present a neuromorphic tactile PVDF-based sensing system for textural features classification, that employs raw signals directly for classification. We first converted raw signals into spikes and then trained recurrent spiking neural networks (RSNNs) using Backpropagation Through Time (BPTT) with surrogate gradients to perform classification. We proposed an optimization method based on tuning the refractory period of the encoding neurons, to explore a potential trade-off between the computational cost and classification accuracy of the RSNN. The proposed method effectively identified two RSNNs with refractory period configurations that achieved a tradeoff between the two evaluation metrics. Following this, we reduced the inference time steps of the selected RSNN during inference using a rate-coding based method. This method succeeded in saving around 26.6% out of the total original time steps. In summary, the proposed system paves the way for establishing an end-to-end neuromorphic approach for tactile textural features classification, through deploying the selected RSNNs on a dedicated neuromorphic hardware device for real-time inferences.

show abstract

Section: B Related Workmentioning

confidence: 99%

Neuromorphic Tactile Sensing System for Textural Features Classification

Al Haj Ali,

Abbass,

Gianoglio

et al. 2024

IEEE Sensors J.

View full text Add to dashboard Cite

show abstract

“…Recently, whit the aim of restoring sensory feedback to amputees in closed loop application there are an increasing interest on the neuromorphic implementation of tactile sensors. The challenge is to encode the tactile information by reproducing the spiking patterns of human primary tactile afferents [71][72][73][74][75]. E-skin [71][72][73] presents a spike encoding following the slow adapting and fast adapting mechanoreceptors in human skin.…”

Section: Sensing In the Peripheral Nervous Systemmentioning

confidence: 99%

“…The challenge is to encode the tactile information by reproducing the spiking patterns of human primary tactile afferents [71][72][73][74][75]. E-skin [71][72][73] presents a spike encoding following the slow adapting and fast adapting mechanoreceptors in human skin. Once the information is encoded, SNNs can be used to extract information about touched objects and surfaces, for example, to detect the orientation of edges [75].…”

Section: Sensing In the Peripheral Nervous Systemmentioning

confidence: 99%

Neuromorphic bioelectronic medicine for nervous system interfaces: from neural computational primitives to medical applications

Donati

Indiveri

2023

Prog. Biomed. Eng.

View full text Add to dashboard Cite

Bioelectronic medicine treats chronic diseases by sensing, processing, and modulating the electronic signals produced in the nervous system of the human body, labeled 'neural signals'. While electronic circuits have been used for several years in this domain, the progress in microelectronic technology is now allowing increasingly accurate and targeted solutions for therapeutic benefits. For example, it is now becoming possible to modulate signals in specific nerve fibers, hence targeting specific diseases. However, to fully exploit this approach it is crucial to understand what aspects of the nerve signals are important, what is the effect of the stimulation, and what circuit designs can best achieve the desired result. Neuromorphic electronic circuits represent a promising design style for achieving this goal: their ultra-low power characteristics and biologically plausible time constants make them the ideal candidate for building optimal interfaces to real neural processing systems, enabling real-time closed-loop interactions with the biological tissue. In this paper, we highlight the main features of neuromorphic circuits that are ideally suited for interfacing with the nervous system and show how they can be used to build closed-loop hybrid artificial and biological neural processing systems. We present examples of neural computational primitives that can be implemented for carrying out computation on the signals sensed in these closed-loop systems and discuss the way to use their outputs for neuro-stimulation. We describe examples of applications that follow this approach, highlight open challenges that need to be addressed, and propose actions required to overcome current limitations.

show abstract

“…Yi, et al 10 Gupta, et al 43 Oddo. et al 16 Sankar, et al 19 Rongala, et al 14 Friedl, et al 44 Liu, et al 15 This work www.nature.com/scientificreports/ thus the contact is not local which may affect the receptive fields.…”

Section: Authormentioning

confidence: 99%

Texture recognition based on multi-sensory integration of proprioceptive and tactile signals

Rostamian

Koolani

Abdollahzade

et al. 2022

Sci Rep

View full text Add to dashboard Cite

The sense of touch plays a fundamental role in enabling us to interact with our surrounding environment. Indeed, the presence of tactile feedback in prostheses greatly assists amputees in doing daily tasks. In this line, the present study proposes an integration of artificial tactile and proprioception receptors for texture discrimination under varying scanning speeds. Here, we fabricated a soft biomimetic fingertip including an 8 × 8 array tactile sensor and a piezoelectric sensor to mimic Merkel, Meissner, and Pacinian mechanoreceptors in glabrous skin, respectively. A hydro-elastomer sensor was fabricated as an artificial proprioception sensor (muscle spindles) to assess the instantaneous speed of the biomimetic fingertip. In this study, we investigated the concept of the complex receptive field of RA-I and SA-I afferents for naturalistic textures. Next, to evaluate the synergy between the mechanoreceptors and muscle spindle afferents, ten naturalistic textures were manipulated by a soft biomimetic fingertip at six different speeds. The sensors’ outputs were converted into neuromorphic spike trains to mimic the firing pattern of biological mechanoreceptors. These spike responses are then analyzed using machine learning classifiers and neural coding paradigms to explore the multi-sensory integration in real experiments. This synergy between muscle spindle and mechanoreceptors in the proposed neuromorphic system represents a generalized texture discrimination scheme and interestingly irrespective of the scanning speed.

show abstract

Spatio-Temporal Encoding Improves Neuromorphic Tactile Texture Classification

Cited by 12 publications

References 34 publications

Neuromorphic Tactile Sensing System for Textural Features Classification

Neuromorphic Tactile Sensing System for Textural Features Classification

Neuromorphic bioelectronic medicine for nervous system interfaces: from neural computational primitives to medical applications

Texture recognition based on multi-sensory integration of proprioceptive and tactile signals

Contact Info

Product

Resources

About