A Compact Butterfly-Style Silicon Photonic–Electronic Neural Chip for Hardware-Efficient Deep Learning

Abstract

The optical neural network (ONN) is a promising hardware platform for next-generation neurocomputing due to its high parallelism, low latency, and low energy consumption. Previous ONN architectures are mainly designed for general matrix multiplication (GEMM), leading to unnecessarily large area cost and high control complexity. Here, we move beyond classical GEMM-based ONNs and propose an optical subspace neural network (OSNN) architecture, which trades the universality of weight representation for lower optical component usage, area cost, and energy consumption. We devise a butterfly-style photonic–electronic neural chip to implement our OSNN with up to 7× fewer trainable optical components compared to GEMM-based ONNs. Additionally, a hardware-aware training framework is provided to minimize the required device programming precision, lessen the chip area, and boost the noise robustness. We experimentally demonstrate the utility of our neural chip in practical image recognition tasks, showing that a measured accuracy of 94.16% can be achieved in handwritten digit recognition tasks with 3 bit weight programming precision.

A hardware-efficient silicon electronic-photonic chip for optical structured neural networks

“…To model the no-idealities such as fabrication errors, programming errors, crosstalk and noise, we employ an AI-assisted, hardware-aware training framework for OSNN training, with details disclosed in our previous work. [7]…”

An overview of brain-like computing: Architecture, applications, and future trends

“…The current development of artificial neuromorphic devices mainly includes two technical routes. One is based on the traditional mature CMOS technology of SRAM or DRAM build (Asghar et al, 2021 ), and the prototype device is volatile in terms of information storage; the other is built based on non-volatile Flexible FLASH devices or new memory devices and new materials (Feng et al, 2021 ; He et al, 2021 ). Non-volatile neuromorphic devices are memristors with artificial neuromorphic characteristics and unique nonlinear properties that have become new basic information processing units that mimic biological neurons and synapses (Yang et al, 2013 ; Prezioso et al, 2015 ).…”

TeMPO: Efficient time-multiplexed dynamic photonic tensor core for edge AI with compact slow-light electro-optic modulator

“…Various integrated photonic tensor core (PTC) designs have been introduced and demonstrated for ultra-fast photonic analog linear operation acceleration. Coherent PTCs that leverage interference and diffraction include MZI arrays, 1 butterfly-style meshes, 2,3 auto-designed photonic circuits, 4 coupler-crossbar array, 5 star-coupler-based design, 6 and metalens-based diffractive PTCs, 7 etc. Besides, to leverage the wavelength-division multiplexing (WDM) technique, there are incoherent multi-wavelength PTCs, e.g., MRR weight bank, [8][9][10][11] PCM crossbar arrays, 12 micro-comb-based computing engine.…”

“…A versatile/generic photonic accelerator based on universal optical linear units is capable of realizing general matrix multiplication (GEMM) and thus directly implementing a wide spectrum of pretrained digital DNNs. Many specialized linear units are not applicable to generic tensor computation since they restrict their matrix expressivity to a subspace of specialized matrices for higher hardware efficiency, e.g., butterfly meshes 3 and tensorized MZI arrays. 15 Besides versatility, photonic computing requires real-time, efficient input tensor encoding with low reconfiguration costs.…”

“…Similarly, many subspace linear unit designs can approximate GEMM operations by cascading more programmable devices but require an even more costly optimization-based approach to map the weight matrix. 3,6 Such a property restricts those designs to only support weight-static linear operations, e.g., fully connected (FC) layers and convolutional (CONV) layers, where weights are pretrained and pre-encoded into the device/circuit transmissions. However, advanced AI models, e.g., Transformer [16][17][18][19][20][21] based on attention operations where both matrix multiplication operands are dynamic, full-range, and general tensors, cannot be efficiently mapped to those weight-static PTCs.…”