A Behavior‐Learned Cross‐Reactive Sensor Matrix for Intelligent Skin Perception

Abstract

Recently, the realization of artificial sensory systems mimicking the biological perception has been intensively pursued for the next generation neuromorphic electronics and humanoid robots. Particularly, an artificial somatosensory system which can emulate the functions of the biological skin and body sensation is considered to have a great potential in achieving highly integrated and neuromorphic sensory network. The biological somatosensory system is a complex sensory network, which is composed of sensory neurons (receptors), neural pathways, and a part of the brain for the perception process. By the sensory receptors such as mechanoreceptors, thermoreceptors, and nociceptors, [1][2][3][4][5][6][7][8][9][10] which are located on or beneath the skin, various environmental stimuli are detected and transmitted to the brain through the neural pathways. This enables the specific sensations such as strain, pressure, temperature, and distortion (flexion/ bending) of the body. In realizing an artificial somatosensory system, however, the integration of a large amount of sensory networks for the individual sensation still remains as a significant challenge, especially in the case of largearea electronic skin (e-skin) devices. For example, it is reported that to realize an e-skin for robotics and prosthetic limbs, an estimated 45 000 mechanoreceptors are needed in about 1.5 m 2 -area devices. [11] Additionally, the number of sensors could increase even further, considering the e-skins to have equivalent numbers of thermoreceptors and nociceptors in the system. Therefore, to fully mimic the biological skin perception over a large-area, a large number of sensory systems with complicated multi-layer architectures would be required as well as a large amount of data associated with their perception processing.In recent research, a new strategy to achieve artificially intelligent perception has been introduced in chemical and gas detection systems by analyzing the different responses recognized from many cross interferences. [12][13][14][15][16][17][18] These cross-reactive sensory systems, inspired by mammalian olfactory and gustatory systems, can simultaneously detect and identify specific responses from a variety of non-specific vapor, liquid elements, and their combinations by analyzing the difference in sensing responses with pattern recognition and machine learning algorithms. [19][20][21][22][23][24][25][26][27] Although these previous advances are noteworthy, Mimicking human skin sensation such as spontaneous multimodal perception and identification/discrimination of intermixed stimuli is severely hindered by the difficulty of efficient integration of complex cutaneous receptor-emulating circuitry and the lack of an appropriate protocol to discern the intermixed signals. Here, a highly stretchable cross-reactive sensor matrix is demonstrated, which can detect, classify, and discriminate various intermixed tactile and thermal stimuli using a machine-learning approach. Particularly, the multimodal perception ability is ...

A Marr's Three‐Level Analytical Framework for Neuromorphic Electronic Systems

“…To distinguish between sensory memory and short‐term memory, memristors possessing second‐order or higher‐order state variables are used so that the first‐stage sensory memory is stored in the internal states of the devices, such as ion distribution, that the device resistances are unchanged, whereas the following stage of short‐term memory is formed by the measurable resistance changes due to the accumulated changes of the internal states. [ 242,243 ] Recent reviews [ 244,245 ] have also surveyed the reports on the integration of sensors, artificial sensory neurons, and synapses [ 84,138,246–261 ] that suggest a trend of the integration of sensing, memory, and computing [ 262,263 ] (Figure 4b). Future neuromorphic electronic systems may also benefit from sensory memory in executing attentive novelty detection tasks and so on.…”

Challenges in Design and Fabrication of Flexible/Stretchable Carbon- and Textile-Based Wearable Sensors for Health Monitoring: A Critical Review

“…Recently, Lee et al reported a behavior-learned cross-reactive sensor array with artificial perception technology based on a machine learning process to detect and distinguish various intermixed stimuli. Unlike most previous wearable sensors focused on the configuration of sensor devices and nanostructured materials engineering technology, the pattern recognition and multimodal perception for intermixed stimuli are possible by adopting a machine learning algorithm [ 161 ]. The combination of carbon- and textile-based wearable sensor arrays with a machine learning algorithm could become a viable approach to realize practical wearable applications that can detect and discriminate multiple stimuli in health monitoring systems.…”

Mechanically robust textile-based strain and pressure multimodal sensors using metal nanowire/polymer conducting fibers

“…Particularly, because the stimuli such as strain and pressure can affect the output signals of each other, precise determination of each stimulation might be difficult unless a proper discrimination of the sensing signals is performed. In fact, by using a machine-learning-based approach, it has been shown that the discrimination of temperature, strain, and pressure could be possible ( Lee et at., 2020 ). However, such approaches may require a considerable number of datasets for training to achieve high accuracy for identifying the mixed stimuli.…”