Channel Tiling for Improved Performance and Accuracy of Optical Neural Network Accelerators

Abstract

Low latency, high throughput inference on Convolution Neural Networks (CNNs) remains a challenge, especially for applications requiring large input or large kernel sizes. 4F optics provides a solution to accelerate CNNs by converting convolutions into Fourier-domain point-wise multiplications that are computationally 'free' in optical domain. However, existing 4F CNN systems suffer from the all-positive sensor readout issue which makes the implementation of a multi-channel, multi-layer CNN not scalable or even impractical.In this paper we propose a simple channel tiling scheme for 4F CNN systems that utilizes the high resolution of 4F system to perform channel summation inherently in optical domain before sensor detection, so the outputs of different channels can be correctly accumulated. Compared to state of the art, channel tiling gives similar accuracy, significantly better robustness to sensing quantization (33% improvement in required sensing precision) error and noise (10dB reduction in tolerable sensing noise), 0.5X total filters required, 10-50X+ throughput improvement and as much as 3X reduction in required output camera resolution/bandwidth. Not requiring any additional optical hardware, proposed channel tiling approach addresses an important throughput and precision bottleneck of high-speed, massively-parallel optical 4F computing systems.

FatNet: High-Resolution Kernels for Classification Using Fully Convolutional Optical Neural Networks

“…Since Moore's law does not affect optical computing, optical accelerators can be used for deep learning, offering advantages such as the high bandwidth of the light beam, high speed, zero resistance, lower energy consumption, and immunity to overheating [12]. There are two main approaches to optical neural networks: free space using spatial light modulators (SLM) [13,14] or silicon photonics approach using Mach-Zehnder interferometers (MZI) [15,16]. Unlike the silicon photonics approach, freespace optics uses wireless light propagation through a medium which can be air, outer space or vacuum.…”

Training on the optical system: local search method

“…By addressing the mismatch between simulated training and experimental setup, the full potential of optical neural networks can be realized. [76][77][78][79][80][81][82] Due to the lack of monitoring for intermediate status, implementing gradient descent algorithms 83 for training optical neural networks on real optical systems can prove to be difficult. [84][85][86][87][88][89][90][91][92][93][94][95] This presents a challenge for achieving optimal performance in real-world scenarios.…”

Black-box simulation method: train the optical model from output

“…There are various methods to address the mismatch degradation in optical systems, [76][77][78][79][80][81][82] such as using misalignment tolerance in neural network parameters or fine-tuning the parameters from real systems. Previous studies attempted to train the system through simulations with the aim of building an optical system that perfectly matched the design, but this resulted in a 20% reduction in classification accuracy compared to the simulation.…”