Mnemonic-opto-synaptic transistor for in-sensor vision system

Abstract

A mnemonic-opto-synaptic transistor (MOST) that has triple functions is demonstrated for an in-sensor vision system. It memorizes a photoresponsivity that corresponds to a synaptic weight as a memory cell, senses light as a photodetector, and performs weight updates as a synapse for machine vision with an artificial neural network (ANN). Herein the memory function added to a previous photodetecting device combined with a photodetector and a synapse provides a technical breakthrough for realizing in-sensor processing that is able to perform image sensing and signal processing in a sensor. A charge trap layer (CTL) was intercalated to gate dielectrics of a vertical pillar-shaped transistor for the memory function. Weight memorized in the CTL makes photoresponsivity tunable for real-time multiplication of the image with a memorized photoresponsivity matrix. Therefore, these multi-faceted features can allow in-sensor processing without external memory for the in-sensor vision system. In particular, the in-sensor vision system can enhance speed and energy efficiency compared to a conventional vision system due to the simultaneous preprocessing of massive data at sensor nodes prior to ANN nodes. Recognition of a simple pattern was demonstrated with full sets of the fabricated MOSTs. Furthermore, recognition of complex hand-written digits in the MNIST database was also demonstrated with software simulations.

A high-dimensional in-sensor reservoir computing system with optoelectronic memristors for high-performance neuromorphic machine vision

“…Therefore, interest in in-sensor computing to address these issues is growing, inspired by the functioning of the human eye, where preprocessing occurs in sensory neurons. [6][7][8][9] Recently, many neuromorphic machine vision systems have been proposed using optoelectronic memristors as the central component of in-sensor computing systems. [10][11][12] Optoelectronic memristors can detect light directly and perform preliminary processing of visual input as in the biological sensory neurons.…”

“…Therefore, interest in in-sensor computing to address these issues is growing, inspired by the functioning of the human eye, where preprocessing occurs in sensory neurons. 6–9…”