Artificial Multisensory Neurons with Fused Haptic and Temperature Perception for Multimodal In‐Sensor Computing

Abstract

The sensory system is a part of the nervous system that processes sensory information, which includes receptors, neural pathways, and the sensory center of the cerebral cortex. In the sensory neural network of humankind, the sensory receptors convert environmental information into potential changes and encode such potential changes into spike trains with neural spike coding in the cell body. Subsequently, interneurons convey the spike trains from the receptors to the cerebral cortex of the brain, where the information is decoded into sensory perceptions for further processing. [1][2][3][4][5][6] Such a structure forms the basis of sensing, preprocessing, and encoding capabilities of the human sensory system. This outperforms digital computers in dealing with a large number of complex tasks, such as perception and real-time sensory data processing. Therefore, it is of great significance to draw inspiration from the human sensory system and develop artificial sensory hardware that can efficiently realize the perception and encoding capabilities, which will provide a promising approach toward e-skin, neurorobotics, and human-machine interaction technologies.

Artificial Multisensory Neuron with a Single Transistor for Multimodal Perception through Hybrid Visual and Thermal Sensing

“…Note that this is the first attempt of hybrid sensing in a single device as previously reported multisensory 1T-neurons included bulky sensing components such as piezoelectric pressure sensor as well as a neuron device. 11,12 Low power consumption of the multisensory 1T-neuron is crucial because energy efficiency is important in neuromorphic hardware. The average power consumption per one spike (P avg ) can be extracted from the following equation 14…”

“…In terms of applications, such a device can be utilized in various fields, including medical diagnosis and robotics, as multimodal sensing can perform various functions that cannot be achieved with only single-modal sensing. Recently, there were reports describing multimodal sensory neurons that could concurrently detect pressure and heat, although the sensory part composed of a bulky piezoelectric pressure sensor was separated from the neuron part that reacted to the temperature. , If the decoupled parts could be merged into a single device, one clear improvement would be miniaturization. Thus far, however, no work has reported the simultaneous detection of light and heat.…”

“…Recently, there were reports describing multimodal sensory neurons that could concurrently detect pressure and heat, although the sensory part composed of a bulky piezoelectric pressure sensor was separated from the neuron part that reacted to the temper- ature. 11,12 If the decoupled parts could be merged into a single device, one clear improvement would be miniaturization. Thus far, however, no work has reported the simultaneous detection of light and heat.…”

Adaptive Hodgkin–Huxley Neuron for Retina‐Inspired Perception

“…The VO 2 's insulator-metal-transition (IMT) is driven by a thermal process, which makes the VO 2 devices quite sensitive to temperature. [39,40] Figure 2b shows the I-V characteristics of the VO 2 TS memristor at temperatures gradually rising from 30 to 40 °C. V th and V hold are extracted from the I-V curve to explain the influence of temperature on the device.…”

“…[43,44] The TS characteristics of VO 2 memristors can be examined and demonstrated by the Mott transition and structural phase transition. [39] Figure 2d shows the comparison of experimental V th and V hold with those gained from the simulation program with integrated circuit emphasis (SPICE) simulation. Figure 2e shows the simulation under temperatures ranging from 30 to 40 °C, to get similar V th variations as experimental results.…”