A large scale photonic matrix processor enabled by charge accumulation

Brückerhoff-Plückelmann, Frank; Bente, Ivonne; Wendland, Daniel; Feldmann, Johannes; Wright, C. David; Bhaskaran, Harish; Pernice, Wolfram H. P.

doi:10.1515/nanoph-2022-0441

Cited by 14 publications

(5 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…At a fundamental level, CNNs are based on linear matrix-vector multiplications. 15 Image edge detection, similarly, involves the use of a kernel that moves across the pixel values of an image, producing a convoluted output. This output results from the multiplication of the flattened image (vector) and the kernel (matrix), as illustrated in Figure 1 a).…”

Section: Convolution Processingmentioning

confidence: 99%

Lithium-niobate photonic integrated circuits for GHz, sub-picojoule/bit optical image processing

Bankwitz,

Dijkstra,

Pradip

et al. 2024

Real-Time Processing of Image, Depth, and Video Information 2024

View full text Add to dashboard Cite

Optical image processing, which capitalizes on the distinctive characteristics of light, facilitates the manipulation of visual data in real-time and at a high speed. This technology is instrumental in performing tasks such as enhancing edges, recognizing patterns, and extracting features, all of which are crucial in fields like medical imaging, surveillance, and industrial automation. In this study, we present the successful demonstration of a photonic integrated circuit (PIC) made of Lithium niobate on insulator, enabling matrix-vector multiplications for image classification. By surpassing an electrical bandwidth of 15 GHz, our experiment showcases the PIC's ability live edge detection and video streaming. Remarkably, its energy efficiency surpasses the limit imposed by electronic systems for each operation by consuming < 10 fJ/bit.

show abstract

Section: Convolution Processingmentioning

confidence: 99%

Lithium-niobate photonic integrated circuits for GHz, sub-picojoule/bit optical image processing

Bankwitz,

Dijkstra,

Pradip

et al. 2024

Real-Time Processing of Image, Depth, and Video Information 2024

View full text Add to dashboard Cite

show abstract

“…From Section IV-A, the stochastic multiplication bit-streams generated by OSMs are guided to a PCA, where they are accumulated to generate a binary output value equivalent to the VDP result. Our PCA is inspired from the time integrating receiver (TIR) design from [37] and the photodetector-based optical-pulse accumulator design from [38]. A PCA circuit, shown in Fig 4(c), has two stages: (i) a stochastic-to-analog conversion stage; (ii) an analog-to-binary conversion stage.…”

Section: Photo Charge Accumulator (Pca)mentioning

confidence: 99%

SCONNA: A Stochastic Computing Based Optical Accelerator for Ultra-Fast, Energy-Efficient Inference of Integer-Quantized CNNs

Vatsavai¹,

Karempudi²,

Thakkar³

et al. 2023

Preprint

View full text Add to dashboard Cite

Convolutional Neural Networks (CNNs) are used extensively for artificial intelligence applications due to their record-breaking accuracy. For efficient and swift hardware-based acceleration, CNNs are typically quantized to have integer input/weight parameters. The acceleration of a CNN inference task uses convolution operations that are typically transformed into vector-dot-product (VDP) operations. Several photonic microring resonators (MRRs) based hardware architectures have been proposed to accelerate integer-quantized CNNs with remarkably higher throughput and energy efficiency compared to their electronic counterparts. However, the existing photonic MRRbased analog accelerators exhibit a very strong trade-off between the achievable input/weight precision and VDP operation size, which severely restricts their achievable VDP operation size for the quantized input/weight precision of 4 bits and higher. The restricted VDP operation size ultimately suppresses computing throughput to severely diminish the achievable performance benefits. To address this shortcoming, we for the first time present a merger of stochastic computing and MRR-based CNN accelerators. To leverage the innate precision flexibility of stochastic computing, we invent an MRR-based optical stochastic multiplier (OSM). We employ multiple OSMs in a cascaded manner using dense wavelength division multiplexing, to forge a novel Stochastic Computing based Optical Neural Network Accelerator (SCONNA). SCONNA achieves significantly high throughput and energy efficiency for accelerating inferences of high-precision quantized CNNs. Our evaluation for the inference of four modern CNNs at 8-bit input/weight precision indicates that SCONNA provides improvements of up to 66.5×, 90×, and 91× in frames-per-second (FPS), FPS/W and FPS/W/mm 2 , respectively, on average over two photonic MRR-based analog CNN accelerators from prior work, with Top-1 accuracy drop of only up to 0.4% for large CNNs and up to 1.5% for small CNNs. We developed a transaction-level, event-driven python-based simulator for the evaluation of SCONNA and other accelerators (https://github.com/uky-UCAT/SC ONN SIM.git).

show abstract

“…Optical MVM is usually implemented using photonic integrated circuits (PICs), such as in. [2][3][4][5][6][7][8] However, to make any computing architecture comparable to existing hardware solutions, it must meet the standards regarding size, robustness, scaling, and accuracy. Many of these are limited by fabrication errors and crosstalk (XT) effects in PICs.…”

Section: Introductionmentioning

confidence: 99%

Physics-informed machine learning for programmable photonic circuits

Teofilovic,

Zibar,

Da Ros

2024

Machine Learning in Photonics

View full text Add to dashboard Cite

Integrated photonic circuits offer a promising platform to implement matrix-vector multiplication in optical feedforward neural networks. The most common implementations rely on thermal phase shifters, which are inevitably susceptible to effects such as thermal and electrical crosstalk. Although deterministic, crosstalk-induced distortions have been challenging to accurately incorporate into physics-based analytical models. Additionally, analog hardware platforms suffer from fabrication deviations, that can have a significant impact on the computing performance, thus limiting scalability in implemented matrix size. In contrast, data-driven modeling techniques have shown to be promising approaches to modeling such circuits, yet they rely on black-box physicsagnostic modeling, require massive and unscalable amounts of training data, and cannot guarantee physically plausible results. Going beyond the data-driven black-box modeling techniques, but still taking advantage of the information captured by the data, we investigate the advantages of using physics-informed machine learning for photonic meshes. We analyze the ability of this approach to provide more accurate, less data-hungry, and physically plausible models for programmable photonic meshes. Moreover, we explore the potential to extract the knowledge from the trained model.

show abstract

A large scale photonic matrix processor enabled by charge accumulation

Cited by 14 publications

References 31 publications

Lithium-niobate photonic integrated circuits for GHz, sub-picojoule/bit optical image processing

Lithium-niobate photonic integrated circuits for GHz, sub-picojoule/bit optical image processing

SCONNA: A Stochastic Computing Based Optical Accelerator for Ultra-Fast, Energy-Efficient Inference of Integer-Quantized CNNs

Physics-informed machine learning for programmable photonic circuits

Contact Info

Product

Resources

About