An artificial sensory neuron with visual-haptic fusion

Abstract

Human behaviors are extremely sophisticated, relying on the adaptive, plastic and event-driven network of sensory neurons. Such neuronal system analyzes multiple sensory cues efficiently to establish accurate depiction of the environment. Here, we develop a bimodal artificial sensory neuron to implement the sensory fusion processes. Such a bimodal artificial sensory neuron collects optic and pressure information from the photodetector and pressure sensors respectively, transmits the bimodal information through an ionic cable, and integrates them into post-synaptic currents by a synaptic transistor. The sensory neuron can be excited in multiple levels by synchronizing the two sensory cues, which enables the manipulating of skeletal myotubes and a robotic hand. Furthermore, enhanced recognition capability achieved on fused visual/haptic cues is confirmed by simulation of a multi-transparency pattern recognition task. Our biomimetic design has the potential to advance technologies in cyborg and neuromorphic systems by endowing them with supramodal perceptual capabilities.

Dense Hydrogen-Bonding Network Boosts Ionic Conductive Hydrogels with Extremely High Toughness, Rapid Self-Recovery, and Autonomous Adhesion for Human-Motion Detection

“…The human body mainly relies on ion channels inside the neurons to transmit information, and an electrolyte in organisms as an ionic conductor plays an essential role [ 14 , 15 ]. This study attempts to construct a highly stretchable ionic conductive hydrogel to imitate the sensing functions of the skin.…”

Organic Synaptic Transistors: The Evolutionary Path from Memory Cells to the Application of Artificial Neural Networks

“…In addition, the development of neuromorphic systems that mimic biological sensory nerves (e.g., auditory, olfactory, motor) by artificial sensory systems are also important because they could enable the advancement of studies on the perception of external environment, perception processing, and movement control to co‐operate systems. [ 37,111,114,125 ]…”

“…Therefore, essentially they are also artificial tactile or visual nervous system. [ 69,114,115 ] Artificial olfactory nervous system and artificial taste nervous system based on organic synaptic transistors are rarely reported. In the future, the many research directions that will take place will be guided according to the development of organic neuromorphological systems, like the auditory system, the olfactory system, the taste system, and the motor control.…”

Artificial Intelligence of Things (AIoT) Enabled Virtual Shop Applications Using Self‐Powered Sensor Enhanced Soft Robotic Manipulator

“…[72] Wan et al have proposed a bimodal artificial sensory neuron based on ionic/electronic hybrid neuromorphic electronics to implement the visual-haptic fusion, providing multidimensional spatial information for the robotic hand. [73] However, many of these mentioned methods are still confronted with challenges in cost effectiveness, energy consumption, compatibility of materials and process. Based on this, the flexible self-powered approaches, including triboelectricity [74,75] and piezoelectricity [76] for stimulus detection, and thermoelectricity [77] and pyroelectricity [78] for temperature sensing, will have great advantages in realizing long-term sustainable IoT intelligent systems because these sensors can generate electrical signals without external electrical bias, i.e., zero-power consumption at the sensor itself, and are made with low-cost fabrication technology.…”