Physics-aware Complex-valued Adversarial Machine Learning in Reconfigurable Diffractive All-optical Neural Network

Abstract

Diffractive optical neural networks have shown promising advantages over electronic circuits for accelerating modern machine learning (ML) algorithms. However, it is challenging to achieve fully programmable all-optical implementation and rapid hardware deployment.Furthermore, understanding the threat of adversarial ML in such system becomes crucial for real-world applications, which remains unexplored. Here, we demonstrate a large-scale, costeffective, complex-valued, and reconfigurable diffractive all-optical neural networks system in the visible range based on cascaded transmissive twisted nematic liquid crystal spatial light modulators. With the assist of categorical reparameterization, we create a physics-aware training framework for the fast and accurate deployment of computer-trained models onto optical hardware. Furthermore, we theoretically analyze and experimentally demonstrate physics-aware adversarial attacks onto the system, which are generated from a complexvalued gradient-based algorithm. The detailed adversarial robustness comparison with conventional multiple layer perceptrons and convolutional neural networks features a distinct statistical adversarial property in diffractive optical neural networks. Our full stack of software and hardware provides new opportunities of employing diffractive optics in a variety of ML tasks and enabling the research on optical adversarial ML.

Physics-aware Differentiable Discrete Codesign for Diffractive Optical Neural Networks

“…The existing hardware system for verifying the demonstrated simulation results from GS-framework is shown in Figure 5 and has been presented by Chen et. al [3]. Briefly, the input images are generated at 532 nm wavelength laser and have the size 100 × 100.…”

“…Our results demonstrate the substantial advantages for DONNs co-design in image classification, particularly when deployed optical devices are limited to low precision. Finally, we verify the proposed approach in our visible range DONNs hardware platform [3] in low precision settings.…”

“…This work studies an efficient and flexible codesign framework that enables physics-aware differentiable discrete mappings from devices to DONN systems via Gumbel-Softmax (GS) [15,19,29], which has been recently experimentally verified on a visible DONNs hardware platform [3]. This approach can overcome the aforementioned limitations with GS enabling fully differentiable discrete mapping regardless of the number representation format, range, or discrete distribution.…”

“…For example, in our DONN system [3], the diffractive layers are implemented with SLMs. Specifically, the SLM is an array of twisted nematic (TN) liquid crystals, which can be twisted to different angles by different applied control voltages, providing different phase modulation for the input light beam.…”