Delocalized Photonic Deep Learning on the Internet's Edge

Abstract

Advances in deep neural networks (DNNs) are transforming science and technology. However, the increasing computational demands of the most powerful DNNs limit deployment on low-power devices, such as smartphones and sensors – and this trend is accelerated by the simultaneous move towards Internet-of-Things (IoT) devices. Numerous efforts are underway to lower power consumption, but a fundamental bottleneck remains due to energy consumption in matrix algebra, even for analog approaches including neuromorphic, analog memory and photonic meshes. Here we introduce and demonstrate a new approach that sharply reduces energy required for matrix algebra by doing away with weight memory access on edge devices, enabling orders of magnitude energy and latency reduction. At the core of our approach is a new concept that decentralizes the DNN for delocalized, optically accelerated matrix algebra on edge devices. Using a silicon photonic smart transceiver, we demonstrate experimentally that this scheme, termed Netcast, dramatically reduces energy consumption. We demonstrate operation in a photon-starved environment with 40 aJ/multiply of optical energy for 98.8% accurate image recognition and <1 photon/multiply using single photon detectors. Furthermore, we show realistic deployment of our system, classifying images with 3 THz of bandwidth over 86 km of deployed optical fiber in a Boston-area fiber network. Our approach enables computing on a new generation of edge devices with speeds comparable to modern digital electronics and power consumption that is orders of magnitude lower.

Optical Neural Networks for Holographic Image Recognition (Invited Paper)

“…In our work, we focus on the complex operations in ONNs driven by the application of holographic image recognition and propose our scheme as cONNs. Compared with other ONNs designed and applied for handwritten digital recognition [31,32] (See SI Section 4 for details), our scheme gives comparable results and manages to process holographic (complex) image information. Using MZI arrays, we perform matrix operations based on image pixel data, and hence the demand for MZI array size is directly related to the size of the input image.…”

Asymptotically fault-tolerant programmable photonics

“…Correlated errors (both corrected and uncorrected) are important because they are tightly connected to the operational bandwidth of the mesh, a critical design parameter for machine learning schemes that require broadband operation, e.g., for parallel processing on wavelength-multiplexed data [50][51][52][53] . All beamsplitters are dispersive, and this dispersion leads to a correlated wavelength-dependent splitter error, which can usually be expanded to first order μ ≈ (dμ/dλ)Δλ.…”

Photonic unsupervised learning variational autoencoder for high-throughput and low-latency image transmission

“…The recent surge of intelligent photonic computing is considered promising to provide a solution to overcome the electronic bottleneck by processing the images directly in the photonic domain ( 19 – 23 ). Diffractive neural networks ( 24 , 25 ), coherent nanophotonic circuits ( 26 ), convolutional accelerators ( 27 , 28 ), fiber computing ( 29 – 31 ), and other optoelectronic devices ( 32 – 36 ) successfully realize parallel photonic neural networks and increase the computational efficiency substantially.…”