Integrated photonic FFT for photonic tensor operations towards efficient and high-speed neural networks

Abstract

<jats:title>Abstract</jats:title><jats:p>The technologically-relevant task of feature extraction from data performed in deep-learning systems is routinely accomplished as repeated fast Fourier transforms (FFT) electronically in prevalent domain-specific architectures such as in graphics processing units (GPU). However, electronics systems are limited with respect to power dissipation and delay, due to wire-charging challenges related to interconnect capacitance. Here we present a silicon photonics-based architecture for convolutional neural networks that harnesses the phase property of light to perform FFTs efficiently by executing the convolution as a multiplication in the Fourier-domain. The algorithmic executing time is determined by the time-of-flight of the signal through this photonic reconfigurable passive FFT ‘filter’ circuit and is on the order of 10’s of picosecond short. A sensitivity analysis shows that this optical processor must be thermally phase stabilized corresponding to a few degrees. Furthermore, we find that for a small sample number, the obtainable number of convolutions per {time, power, and chip area) outperforms GPUs by about two orders of magnitude. Lastly, we show that, conceptually, the optical FFT and convolution-processing performance is indeed directly linked to optoelectronic device-level, and improvements in plasmonics, metamaterials or nanophotonics are fueling next generation densely interconnected intelligent photonic circuits with relevance for edge-computing 5G networks by processing tensor operations optically.</jats:p>

Nonvolatile programmable silicon photonics using an ultralow-loss Sb 2 Se 3 phase change material

“…The use of nonvolatile programmable MZI with large optical phase shifts exceeding 2π is of interest. Referred to as optical true time delay (OTTD), programmable optical path lengths are of interest in microwave photonics ( 24 ), optical fast Fourier transforms ( 25 ), Fourier transform sensors ( 26 ), and integrated quantum circuits ( 27 ). In microwave photonics and emerging terahertz (6G) applications ( 28 , 29 ), OTTDs are used for beam formation using phase array antennas and in optical communications for signal synchronization, equalization, buffering, and time division multiplexing.…”

Silicon-based optoelectronics for general-purpose matrix computation: a review

“…5(e) is the on-chip Cooley-Turkey method FT executing the CONV on the order of tens of picoseconds short. 62 Once the FT device is realized, then the "4f" CONV system can be realized by using a cascade of two photonic FT devices with a phase and amplitude filter mask in between. The limitation of FT-based CONV is that FT devices typically take up large on-chip space due to the need for free-space propagation.…”

“…(d) Compact surface plasmon polaritons device for FT 61. (e) CONV based on Cooley-Tukey method FT 62.…”

Planar Fourier optics for slab waveguides, surface plasmon polaritons, and 2D materials

“…In recent work in the field of photonic computing, some of the familiar models from Fourier optics in 3D free-space have been adapted to the setting of slab (or planar) waveguides [12][13][14][15][16][17][18][19][20],…”