No abstract

The incorporation of high-performance optoelectronic devices into photonic neuromorphic processors can substantially accelerate computationally intensive matrix multiplication operations in machine learning (ML) algorithms. However, the conventional designs of individual devices and system are largely disconnected, and the system optimization is limited to the manual exploration of a small design space. Here, a device-system end-to-end design methodology is reported to optimize a free-space optical general matrix multiplication (GEMM) hardware accelerator by engineering a spatially reconfigurable array made from chalcogenide phase change materials. With a highly parallelized integrated hardware emulator with experimental information, the design of unit device to directly optimize GEMM calculation accuracy is achieved by exploring a large parameter space through reinforcement learning algorithms, including deep Q-learning neural network, Bayesian optimization, and their cascaded approach. The algorithm-generated physical quantities show a clear correlation between system performance metrics and device specifications. Furthermore, physics-aware training approaches are employed to deploy optimized hardware to the tasks of image classification, materials discovery, and a closed-loop design of optical ML accelerators. The demonstrated framework offers insights into the end-to-end and co-design of optoelectronic devices and systems with reduced human supervision and domain knowledge barriers.

The incorporation of high-performance optoelectronic devices into photonic neuromorphic processors can substantially accelerate computationally intensive operations in machine learning (ML) algorithms. However, the conventional device design wisdom is disconnected with system optimization. We report a device-system co-design methodology to optimize a freespace optical general matrix multiplication (GEMM) hardware accelerator by engineering a spatially reconfigurable array made from chalcogenide phase change materials. With a highly-parallelized hardware emulator constructed based on experimental information, we demonstrate the design of unit device by optimizing GEMM calculation accuracy via reinforcement learning, including deep Q-learning neural network, Bayesian optimization, and their cascaded approach, which show a clear correlation between system performance metrics and physical device specifications. Furthermore, we employ physics-aware training approaches to deploy optimized hardware to the tasks of image classification, materials discovery, and a closed-loop design of optical ML accelerators. The demonstrated framework offers insights into the co-design of optoelectronic devices and systems with reduced humansupervision and domain-knowledge barriers.

In this second part of a two-part study, we extend to nonlinear synaptic responses a new framework, called Response Surfaces (RSs), for analyzing, designing, and visualizing spiking neurons and networks. An RS is the transfer function between input and output firing times of a spiking neuron and shows all the patterns of input spike times that fire a neuron at a given time. Here in Part II, we present four mathematically tractable functions to model more realistic post-synaptic-potential waveforms, and we build on the linear RS framework of Part I to create a nonlinear RS framework. We then discuss the qualitative differences between the linear and nonlinear RSs frameworks and revisit problems of Part I using the nonlinear RSs framework: graphing the transfer function of a nonlinear spiking neuron, designing an efficient spiking-XOR gate, analyzing phase-tracking in recurrent SNNs, and developing transfer functions for calculating the output firing times of a spiking neuron given a set of input firing times. For the nonlinear RS framework, we show that the output firing time of a nonlinear spiking neuron is equivalent to the center-ofmass of input spike times (acting as positions) and weights (acting as masses) plus a fixed delay plus a secondorder correction. The second-order centroid correction is one of the main differences between the linear and nonlinear RS frameworks. For recurrent networks, the nonlinear RS framework also shows a more stable behavior compared to linear SNNs, owing to the smooth shapes of nonlinear post-synaptic-potential waveforms.