Integrated photonic metasystem for image classifications at telecommunication wavelength

Abstract

Miniaturized image classifiers are potential for revolutionizing their applications in optical communication, autonomous vehicles, and healthcare. With deep diffractive neuron networks trained subwavelength structures, we demonstrate image recognitions by a passive silicon photonic metasystem. The metasystem implements high-throughput vector-by-matrix multiplications, enabled by 103 passive subwavelength phase shifters as weight elements in 1 mm2 footprint. The large weight matrix size incorporates the fabrication variation related uncertainties, and thus the pre-trained metasystem can perform machine learning tasks without post-tuning. A 15-pixel spatial pattern classifier reaches near 90% accuracy with femtosecond inputs. The metasystem’s superior parallelism (1015 bit/s) dramatically expand data processing capability of photonic integrated circuits, towards next generation low latency and low power photonic accelerators compatible with complementary metal-oxide-semiconductor manufacturing.

Photonic machine learning with on-chip diffractive optics

“…First, through using more advanced machining equipment, which can fundamentally reduce the error caused by chip processing. Second, the random phase offset with uniform distribution within the interval, such as (0,0.5π), can be introduced to each part during the training stage, such as the signal loading and fabrication stages, to improve the system’s robustness against nanofabrication variations and phase fluctuations in measurement 33 . Last but not least, it is extraordinarily significant to further improve the resolution of the testing instrument and the stability of the testing environment to ensure that the error caused by the testing process is minimized.…”

“…However, to solve complex tasks in a timely manner, ANNs require massive amounts of resources, both regarding computing speed and energy consumption. In recent decades, optical neural networks (ONNs) have garnered tremendous interest, because of their advantages of low power consumption and ultrahigh computing bandwidth, which are unrivaled by their electronic counterparts 18 – 33 . Several implementations of ONNs have been proposed, including a coherent approach based on an integrated Mach‒Zehnder interferometer (MZI) mesh 18 , 24 , 25 , 31 , wavelength division multiplexing (WDM) processing with microring modulators, and programmable routing enabled by a phase-change material (PCM) 20 .…”

Intelligent meta-imagers: From compressed to learned sensing

“…To date, such programmable diffractive layers have not yet been endowed with non-linear features that would be needed for complicated processing tasks. While the compatibility of the concept with CMOS technology for operation at near-infrared frequencies has been demonstrated for static linear systems, 230,231 the integrability of cascaded diffractive layers at microwave frequencies appears in general, difficult due to their inherent bulkiness and reliance on free-space propagation. 229,232 Applied Physics Reviews Most recent works on cascaded diffractive layers can in fact be understood as all-analog sensing pipelines.…”

Deep learning-enabled compact optical trigonometric operator with metasurface

“…Compared to conventional Fourier-based optical computing devices, metasurfaces can achieve modulation of the EM profile within the sub-wavelength thickness, which facilitates the miniaturization of photonic signal processors in volume. The superiority of such a metasurface-based optical computing strategy has been demonstrated in various optical signal processing scenarios, such as spatial differentiation, integration, and convolution [19][20][21], Laplacian operation [22], image processing and classifications [23,24], solving equations [25,26]. However, the proposed metasurface-based optical computing works have mainly dealt with spatialdomain filtering and frequency-domain filtering, while lack of the solutions for optical function operations with specific numerical inputs.…”