On the Reliability of Computing-in-Memory Accelerators for Deep Neural Networks

Yan, Zheyu; Hu, Xiaobo Sharon; Shi, Yiyu

doi:10.1007/978-3-031-02063-6_9

Cited by 6 publications

(2 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Depending on the technology, static errors can correspond to programming failures, or temporal fluctuations of the device with a timescale higher than the inference time, such as conductance relaxation for RRAMs [25], or drift for PCRAMs [26]. Dynamic errors can correspond to the inherent noisy behavior of analog devices [10], [17], [27]. In addition, read operations on FeRAMs are destructive [4] and hence errors are always dynamic as the device is re-programmed after each read.…”

Section: Multi-level Non-volatile Memories Modelmentioning

confidence: 99%

See 1 more Smart Citation

Improving the Robustness of Neural Networks to Noisy Multi-Level Non-Volatile Memory-based Synapses

Dampfhoffer

Lopez

Mesquida

et al. 2023

2023 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

The implementation of Artificial Neural Networks (ANNs) using analog Non-Volatile Memories (NVMs) for synaptic weights storage promises improved energy-efficiency and higher density compared to fully-digital implementations. However, NVMs are prone to variability, resulting in a degradation of the accuracy of ANNs. In this paper, a general methodology to evaluate and enhance the accuracy of neural networks implemented with non-ideal multi-level NVMs is presented. A hardware fault model distinguishing two types of errors, namely static and dynamic, capturing the variability of NVMs is proposed. Considering various neural networks, it is shown that error-aware training highly increases the robustness to errors compared to a standard, error-agnostic, training. Moreover, Recurrent NNs (RNNs) and Spiking NNs (SNNs) are found to be inherently more robust to dynamic errors than Convolutional NNs (CNNs). In addition, new insights on the adaptability of neural networks to noisy multi-level NVMs are presented, which could further improve their robustness in this context. The methodology aims at providing tools for hardware-software co-design, paving the way for a broader use of multi-level NVM-based synapses.

show abstract

Section: Multi-level Non-volatile Memories Modelmentioning

confidence: 99%

“…Therefore, enhancing the robustness of neural networks to noisy multi-level weights is essential to achieve accurate and efficient hardware implementations of ANNs. In addition, variability in NVMs comes from different sources and results in different types of errors, which can have a different impact on the accuracy of ANNs [9], [10].…”

Section: Introductionmentioning

confidence: 99%