Nanomaterial-Based Synaptic Optoelectronic Devices for In-Sensor Preprocessing of Image Data

Abstract

With the advance in information technologies involving machine vision applications, the demand for energy- and time-efficient acquisition, transfer, and processing of a large amount of image data has rapidly increased. However, current architectures of the machine vision system have inherent limitations in terms of power consumption and data latency owing to the physical isolation of image sensors and processors. Meanwhile, synaptic optoelectronic devices that exhibit photoresponse similar to the behaviors of the human synapse enable in-sensor preprocessing, which makes the front-end part of the image recognition process more efficient. Herein, we review recent progress in the development of synaptic optoelectronic devices using functional nanomaterials and their unique interfacial characteristics. First, we provide an overview of representative functional nanomaterials and device configurations for the synaptic optoelectronic devices. Then, we discuss the underlying physics of each nanomaterial in the synaptic optoelectronic device and explain related device characteristics that allow for the in-sensor preprocessing. We also discuss advantages achieved by the application of the synaptic optoelectronic devices to image preprocessing, such as contrast enhancement and image filtering. Finally, we conclude this review and present a short prospect.

Flexible and Stretchable Light-Emitting Diodes and Photodetectors for Human-Centric Optoelectronics

“…For instance, human vision senses light signals and efficiently handles the data via the neural process in the visual cortex of the brain to acquire, memorize, and process visual information, enabling the classification and recognition of object size and distance. In contrast, conventional imaging devices perform visual tasks by integrating the image sensor with complementary backend electronics, including central processing units (CPUs), memory devices, and graphics processing units (GPUs), which results in several drawbacks in terms of computing speed and power consumption. , To realize efficient vision systems mimicking the human visual process, the imaging device that enables both image acquisition and data processing has attracted great attention in recent years. ,− …”

Nanomaterial-Based Artificial Vision Systems: From Bioinspired Electronic Eyes to In-Sensor Processing Devices

“…104 Image signal processing (ISP) techniques have improved data transfer and processing performances by preprocessing image data before it reaches to the post-processing stage. 20 One notable ISP technique is to emphasize key features of target objects through contrast enhancement (Figure 6b, (i)). This enables artificial intelligence to recognize a target object with high accuracy.…”

“…For example, to ensure comprehensive observation of the entire region-of-interest without blind spots, it is important to achieve panoramic imaging capability allowing clear capture of all objects located across wide ranges of field-of-views (FoVs) and distances (Figure b). , In the detection of target objects, it is necessary to acquire detailed information with high visual clarity by focusing and magnifying the objects . Nevertheless, environmental constraints and uncertainties, such as uneven sunlight, external medium, and changeable weather conditions, often deteriorate contrast and clarity, and thus prohibit high-quality image acquisition (Figure c). − After obtaining image data, including the target object, it is imperative to recognize it efficiently and swiftly. , However, this procedure involves processing a large amount of image data and needs significant power consumption and processing time, , which are incompatible with situations of rapidly moving mobile robots . Thus, energy-efficient and time-efficient image data processing is crucial to enhance the decision-making capability of a mobile robot (Figure d). − …”

“…16−18 After obtaining image data, including the target object, it is imperative to recognize it efficiently and swiftly. 19,20 However, this procedure involves processing a large amount of image data and needs significant power consumption and processing time, 21,22 which are incompatible with situations of rapidly moving mobile robots. 7 Thus, energy-efficient and time-efficient image data processing is crucial to enhance the decision-making capability of a mobile robot (Figure 1d).…”

“…Achieving multifunctionality for application-specific imaging capabilities also demands the integration of bulky and heavy optical components into the camera module, , which can significantly reduce the mobility of the robots. Furthermore, the primary functions of conventional cameras prioritize image acquisition, necessitating additional high-performance processors and large energy storage devices, , neither of which are suitable for the desired compactness and lightweight for mobile robots.…”