Analyzing the Effects of Noise and Variation on the Accuracy of Analog Neural Networks

Janke, Devon; Anderson, David V.

doi:10.1109/mwscas48704.2020.9184644

Cited by 6 publications

(1 citation statement)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Analog electronic processors based on binary neural networks showed the impact of comparator noise on accuracy 17 . Analog electronic multilayer perceptrons are sensitive on temperature and voltage fluctuations as source of noise 18 . The central importance of noise for future NN hardware solutions is therefore moving into focus, work remains restricted on particular hardware [19][20][21][22] rather than on the general principles of noise in NNs.…”

mentioning

confidence: 99%

Fundamental aspects of noise in analog-hardware neural networks

Semenova

Porté

Andréoli

et al. 2019

Chaos: An Interdisciplinary Journal of Nonlinear Science

View full text Add to dashboard Cite

We study and analyze the fundamental aspects of noise propagation in recurrent as well as deep, multi-layer networks. The main focus of our study are neural networks in analogue hardware, yet the methodology provides insight for networks in general. The system under study consists of noisy linear nodes, and we investigate the signal-to-noise ratio at the network's outputs which is the upper limit to such a system's computing accuracy. We consider additive and multiplicative noise which can be purely local as well as correlated across populations of neurons. This covers the chief internal-perturbations of hardware networks and noise amplitudes were obtained from a physically implemented recurrent neural network and therefore correspond to a real-world system. Analytic solutions agree exceptionally well with numerical data, enabling clear identification of the most critical components and aspects for noise management. Focusing on linear nodes isolates the impact of network connections and allows us to derive strategies for mitigating noise. Our work is the starting point in addressing this aspect of analogue neural networks, and our results identify notoriously sensitive points while simultaneously highlighting the robustness of such computational systems.

show abstract

mentioning

confidence: 99%