All Two‐dimensional Integration‐Type Optoelectronic Synapse Mimicking Visual Attention Mechanism for Multi‐Target Recognition

Abstract

The human visual attention mechanism enables them to rapidly perceive important information and objects in a complex external scene; this effectively solves the problems of data redundancy, low-resolution images, and substantial computing resources. The process by which the attention system reconstructs the visual information can be considered as integrating internal attention signals with external visual details in the postsynaptic neuron. However, electronic devices that simulate visual attention modulation by incorporating device characteristics into neuromorphic vision systems (NVSs) to achieve visual attention behavior are rarely reported. Herein, a synapse device that integrates optical and electrical stimulation is designed using ReS 2 /hBN/monolayer graphene heterojunction to mimic attention regulation and integrate multiple neuron signals successfully. The synapse array can imitate perceptual learning of the human visual system (HVS) to realize visual preprocessing, such as image contrast improvement and weak signal enhancement at the sensory terminal, and overcome data redundancy. Moreover, by applying gate voltage pulses, electric-tunable synaptic plasticity is successfully observed, attributed to the carrier trapping and de-trapping mechanism in the floating layer. Attention stabilization, fluctuation, distraction, and reinforcement are exhibited, simulating the attention behaviors of the HVS. Thus, an NVS with attention mechanism is established depending on the optoelectronic hybrid synaptic plasticity of the device, which successfully mimics the HVS to perform a multi-target recognition task. Furthermore, the effect of device defects on the NVS is rarely evaluated, in which a method is provided to analyze the application results of the NVS when considering uniformity and fault rate. This study may provide new inspiration for developing neuromorphic vision systems for autonomous driving and brainwave control in the future.

Manufacturing of graphene based synaptic devices for optoelectronic applications

“…Visual attention mechanism allows brain focusing on important information and ignoring unimportant information, facilitating the efficient multi-target recognition. Chen et al [157] designed an floating-gate transistor with ReS 2 /hBN/graphene heterojunction, where ReS 2 , hBN, and graphene was employed as semiconductor channel, dielectric, and charge trapping layer, respectively. The device can achieve light-tunable synaptic plasticity under optical stimuli and electrical synaptic plasticity by applying gate voltage.…”

Energy-Efficient ReS2-Based Optoelectronic Synapse for 3D Object Reconstruction and Recognition

“…Inspired by the human visual system (HVS) with low redundancy, low latency, high dynamics, and adaptation, a neuromorphic vision system (NVS) based on optoelectronic synapses is being considered to provide an effective path to break through the bottleneck of the conventional artificial vision system (AVS). − Currently, NVSs exhibit powerful visual information preprocessing capabilities, such as image contrast enhancement, , image denoising, , feature extraction, , pattern recognition, − and motion detection, − thereby addressing the main problem of the traditional AVS, which demands vast computing resources to process gigantic redundant data . In situ storage has also been successfully achieved at the sensor end owing to optoelectronic synapses that mimic visual learning and memory, thereby considerably reducing the dependence on system memory. − Moreover, optoelectronic synapses with visual adaptation can actively adjust NVSs to match new visual tasks in varying environments and successfully adapt to bright/dark conditions and various visual angles. − To achieve ultrafast vision, based on the nonlinear photoresponse and positive/negative photoconductive behaviors of optoelectronic synapses, NVSs can handle inputting visual information at the sensors end in real time via reservoir computing − and convolution operation, which can address high transmission delay limitation of the AVS with separated architectures. − …”

“…Inspired by the human visual system (HVS) with low redundancy, low latency, high dynamics, and adaptation, a neuromorphic vision system (NVS) based on optoelectronic synapses is being considered to provide an effective path to break through the bottleneck of the conventional artificial vision system (AVS). 1−3 Currently, NVSs exhibit powerful visual information preprocessing capabilities, such as image contrast enhancement, 4,5 image denoising, 6,7 feature extraction, 8,9 pattern recognition, 10−12 and motion detection, 13−15 thereby addressing the main problem of the traditional AVS, which demands vast computing resources to process gigantic redundant data. 16 In situ storage has also been successfully achieved at the sensor end owing to optoelectronic synapses that mimic visual learning and memory, thereby considerably reducing the dependence on system memory.…”

Ga2O3 Bipolar Heterojunction-Based Optoelectronic Synapse Array with Visual Attention

“…18 Recently, the introducing attention mechanism has been demonstrated in an optoelectronic synapse transistor based on a ReS 2 /hBN/monolayer graphene heterojunction, which allows for selective accentuation of relevant information that enhances the capability in multitarget recognition. 19 While significant progress has been made, existing AVNs still grapple with challenges in energy efficiency and robustness, especially when compared to the extraordinary feature extraction capabilities of the biological vision system.…”

“…For instance, intelligent tasks, such as handling complex image processing like visual adaptation, − visual pattern recognition, − and collision avoidance, were mimicked by these devices for enhancing recognition accuracy. , Especially, a hardware-level photoelectronic reservoir computing (RC) system, combining photosynapse and a non-volatile memristor array, has also been innovatively developed by Zhang et al to recognize the latent fingerprint with in-sensor and parallel in-memory computing . Recently, the introducing attention mechanism has been demonstrated in an optoelectronic synapse transistor based on a ReS 2 /hBN/monolayer graphene heterojunction, which allows for selective accentuation of relevant information that enhances the capability in multi-target recognition . While significant progress has been made, existing AVNs still grapple with challenges in energy efficiency and robustness, especially when compared to the extraordinary feature extraction capabilities of the biological vision system.…”