Nonlinear Optical Joint Transform Correlator for Low Latency Convolution Operations

Abstract

Convolutions are one of the most relevant operations in artificial intelligence (AI) systems. High computational complexity scaling poses significant challenges, especially in fast-responding networkedge AI applications. Fortunately, the Convolution Theorem can be executed 'on-the-fly' in the optical domain via a joint transform correlator (JTC) offering to fundamentally reduce the computational complexity. Nonetheless, the iterative two-step process of a classical JTC renders them unpractical. Here we introduce a novel implementation of an optical convolution-processor capable of near-zero latency by utilizing all-optical nonlinearity inside a JTC, thus minimizing electronic signal or conversion delay. Fundamentally we show how this nonlinear auto-correlator enables reducing the high O(n 4 ) scaling complexity of processing two-dimensional data to only O(n 2 ). Moreover, this optical JTC processes millions of channels in time-parallel, ideal for large-matrix machine learning tasks. Exemplary utilizing the nonlinear process of four-wave mixing, we show light processing performing a full convolution that is temporally limited only by geometric features of the lens and the nonlinear material's response time. We further discuss that the all-optical nonlinearity exhibits gain in excess of > 10 3 when enhanced by slow-light effects such as epsilon-near-zero. Such novel implementation for a machine learning accelerator featuring low-latency and non-iterative massive data parallelism enabled by fundamental reduced complexity scaling bears significant promise for network-edge, and cloud AI systems.

Optical Correlators for Cryptosystems and Image Recognition: A Review

“…In an NL-JTC, for controlling the contrast between the correlation peaks at the output, the Fourier plane is provided with a nonlinear intensity threshold. One of the possibilities for obtaining a NL-JTC is to benefit from the nonlinear process of four-wave mixing, to create a JTC based on this process [ 46 ].…”

“…In the domain of artificial intelligence (AI), recently a novel application of the nonlinear JTC correlator was proposed by George et al [ 46 ]. Since the traditional JTC architecture is iterative and thus impractical for AI systems, the paper introduced a processor capable of executing the convolution in the optical domain, with a very low latency (close to zero), benefitting from the nonlinearity of the correlator.…”

“…The JTC correlator proposed by George et al [ 46 ], using nonlinear materials, might prove useful, besides the optical processing domain, in future applications such as edge computing, machine vision or in technologies for smart homes. As concerns the machine learning domain, this optical correlator could be able to bring about a revolution in fields such as recognition of speech, translation of foreign languages, analysis of images in medicine, autonomous vehicles, robots, and many others.…”

Reconfigurable focusing and defocusing meta lens using Sb2Se3 phase change material

“…Due the small volume of the metalens compared to traditional converging lens, the active tunning approach could be used in many nanophotonic applications, such as novel combining beam switching 20 and lensing 24 , dual-functional devices 25 , and active holography. More important, the realization of non-volatile reconfigurable metalens pave the way for the development of integrated all optical free space convolutional neuromorphic networks [26][27][28][29][30][31][32][33] . For which the tunable meta lens could be used to replace the bulky lens and DMD in the complicated light path for the development of integrated version.…”

“…For which the tunable meta lens could be used to replace the bulky lens and DMD in the complicated light path for the development of integrated version. Compare to other proposed on-chip integrated photonic hardware accelerator, in memory computational photonic tensor core 23,27,31,[34][35][36][37][38][39][40][41][42][43][44] , all optical convolution solution provides alternative solution with higher data processing rate and lower power consumption 45,46 . With the combine implementation of high performance electrooptic modulators [47][48][49][50][51][52][53][54][55][56] , photodetectors [57][58][59] , signal converters 60 and the reconfigurable metalens, a fully integrated all optical neuromorphic architecture could be realized soon.…”