Dynamic Precision Analog Computing for Neural Networks

Garg, Sahaj; Lou, Joe; Jain, Anirudh; Nahmias, Mitchell A.

doi:10.1109/jstqe.2022.3218019

Cited by 18 publications

(8 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As an example, we study a photonics-based RNS analog accelerator design that is thermal and shot noise-limited. The noise can be modeled as a Gaussian distribution that is additive to the output value, i.e., Σ j x j w j + N ð0,1Þσ noise for a dot product 31 . Many other noise sources present in various analog designs can be represented using a similar framework.…”

Section: Rrns For Fault Tolerancementioning

confidence: 99%

A blueprint for precise and fault-tolerant analog neural networks

Demirkiran,

Nair,

Bunandar

et al. 2024

Nat Commun

View full text Add to dashboard Cite

Analog computing has reemerged as a promising avenue for accelerating deep neural networks (DNNs) to overcome the scalability challenges posed by traditional digital architectures. However, achieving high precision using analog technologies is challenging, as high-precision data converters are costly and impractical. In this work, we address this challenge by using the residue number system (RNS) and composing high-precision operations from multiple low-precision operations, thereby eliminating the need for high-precision data converters and information loss. Our study demonstrates that the RNS-based approach can achieve ≥99% FP32 accuracy with 6-bit integer arithmetic for DNN inference and 7-bit for DNN training. The reduced precision requirements imply that using RNS can achieve several orders of magnitude higher energy efficiency while maintaining the same throughput compared to conventional analog hardware with the same precision. We also present a fault-tolerant dataflow using redundant RNS to protect the computation against noise and errors inherent within analog hardware.

show abstract

Section: Rrns For Fault Tolerancementioning

confidence: 99%

A blueprint for precise and fault-tolerant analog neural networks

Demirkiran,

Nair,

Bunandar

et al. 2024

Nat Commun

View full text Add to dashboard Cite

show abstract

“…error of σ rms = 0.005, corresponding to at least 7 bits of precision, higher than the ∼5 bits usually required for DNN inference. 24,25 As confirmation, we run a pre-trained DNN on a benchmark image classification task (MNIST) using the NetCast hardware, obtaining a classification accuracy of 98.8% (Fig. 5(d)) when employing 3 THz of optical bandwidth over the deployed fiber.…”

Section: Netcast: Wdm-powered Photonic Edge Computingmentioning

confidence: 99%

Multiplexing methods for scaling up photonic logic

Hamerly

Bandyopadhyay

Sludds

et al. 2023

AI and Optical Data Sciences IV

View full text Add to dashboard Cite

Existing photonic matrix processers are too small to tackle relevant problems. Here, I review our group's recent work on scaling up analog photonic platforms. This work includes iterative advances to old approaches (accurate methods to calibrate MZI meshes), experimental demonstrations of recent proposals (a VCSEL array-based coherent detection ONN and a single-shot ONN based on reconfigurable free-space optical fan-out and weighting) and entirely new architectures (WDM-powered and RF-photonic fiber circuits for edge computing). The lessons learned from studying this diverse array of approaches helps inform the future development of photonic hardware for computation.

show abstract

“…Dynamic precision analog computing for NN was proposed in Ref. [240]. It lies in repeating operations and averaging the result, decreasing the impact of noise.…”

Section: Noise In Analog Optical Computingmentioning

confidence: 99%

“…The significant advance is achieving capacity exceeding 1.5 × 10 10 optical nodes, which enables large‐scale applications. In another work, [ 240 ] authors realized an optical NN to simulate inference at an optical energy consumption of 2.7 aJ per MAC for computer vision model Resnet50 (Residual Network) and 1.6 aJ per MAC for natural language processing model BERT (Bidirectional Encoder Representations from Transformers) with little accuracy degradation. The third example [ 241 ] demonstrated a realization of a complicated generative network on the basis of a photonic computing core consisting of an array of programmable phase‐change metasurface mode converters.…”

Section: Mathematical Formulation Of Applicationsmentioning

confidence: 99%

Analog Photonics Computing for Information Processing, Inference, and Optimization

Nikita

Berloff

2023

Adv Quantum Tech

View full text Add to dashboard Cite

This review presents an overview of the current state‐of‐the‐art in photonics computing, which leverages photons, photons coupled with matter, and optics‐related technologies for effective and efficient computational purposes. It covers the history and development of photonics computing and modern analogue computing platforms and architectures, focusing on optimization tasks and neural network implementations. The authors examine special‐purpose optimizers, mathematical descriptions of photonics optimizers, and their various interconnections. Disparate applications are discussed, including direct encoding, logistics, finance, phase retrieval, machine learning, neural networks, probabilistic graphical models, and image processing, among many others. The main directions of technological advancement and associated challenges in photonics computing are explored, along with an assessment of its efficiency. Finally, the paper discusses prospects and the field of optical quantum computing, providing insights into the potential applications of this technology.

show abstract

Dynamic Precision Analog Computing for Neural Networks

Cited by 18 publications

References 49 publications

A blueprint for precise and fault-tolerant analog neural networks

A blueprint for precise and fault-tolerant analog neural networks

Multiplexing methods for scaling up photonic logic

Analog Photonics Computing for Information Processing, Inference, and Optimization

Contact Info

Product

Resources

About