In situ optical backpropagation training of diffractive optical neural networks

Abstract

Training an artificial neural network with backpropagation algorithms to perform advanced machine learning tasks requires an extensive computational process. This paper proposes to implement the backpropagation algorithm optically for in situ training of both linear and nonlinear diffractive optical neural networks, which enables the acceleration of training speed and improvement in energy efficiency on core computing modules. We demonstrate that the gradient of a loss function with respect to the weights of diffractive layers can be accurately calculated by measuring the forward and backward propagated optical fields based on light reciprocity and phase conjunction principles. The diffractive modulation weights are updated by programming a high-speed spatial light modulator to minimize the error between prediction and target output and perform inference tasks at the speed of light. We numerically validate the effectiveness of our approach on simulated networks for various applications. The proposed in situ optical learning architecture achieves accuracy comparable to in silico training with an electronic computer on the tasks of object classification and matrix-vector multiplication, which further allows the diffractive optical neural network to adapt to system imperfections. Also, the self-adaptive property of our approach facilitates the novel application of the network for all-optical imaging through scattering media. The proposed approach paves the way for robust implementation of large-scale diffractive neural networks to perform distinctive tasks all-optically.

Nanoprinted high-neuron-density optical linear perceptrons performing near-infrared inference on a CMOS chip

“…Our approach is based on static elements realised with linear materials. Dynamicity and optical non-linearities are elements essential for the in situ training of optical neural networks 18 , 43 . While reconfigurability can be incorporated into MLDs using compact reconfigurable optical elements 44 – 48 and metamaterials 49 – 51 , non-linear materials, e.g., chalcogenide glasses 32 or ferroelectric thin films 33 , can be used to include non-linearities, thus enabling closed-loop machine learning with the equivalent of a non-linear activation function to further improve the MLD performance 52 .…”

Research progress in optical neural networks: theory, applications and developments

“…In addition, the two-layer AONN [34], designed by the Hong Kong team, proposed a special nonlinear activation function based on electromagnetic induced transparency (EIT)--a photo-induced quantum interference effect between atomic transitions, in laser-cooled atoms with electromagnetic induced transparency. The EIT nonlinear optical activation function is implemented by laser-cooled 85 Rb atoms in the dark-line two-dimensional magneto-optical trap (MOT), as shown in Fig. 10 In January 2020, a nonlinear activation function structure of the optical neural network was proposed [83], which achieves the optical-to-optical nonlinearity by converting a small part of the optical input power into voltage.…”

“…In June 2020, Dai Qionghai team proposed a SLM cascaded neural network, which uses 4f system and SLM to achieve optical field measurement, and utilizes the error measurement module to realize network training [85], as shown in Fig. 11(b).…”

“…11 a Possible solutions for optical logic gate cascading and other possible implementation platforms [26]. b Describes a SLM cascade neural network, and uses 4f system and SLM to realize optical field measurement, thereby carrying out optical training of diffraction ONN [85] gradient descent method to update the parameters, and constructed a two-layer fully connected neural network for speech recognition experiment. The recognition accuracy is only 76.7%, while equivalent electronic neural network can achieve accuracy of 91.7%.…”

Quadrature Photonic Spatial Ising Machine

“…However, although this proposal, intrinsically behaving as an accelerator of vectormatrix multiplication, provides flexibility for problem configuration, it is hindered by large number of iterations and integration scalability due to imperfect components [17,21]. Diffractive component has shown very unique feature due to spatial multiplexing and will present profound influence on large-scale optical computing [18,22,23,24,25,26].…”