Measuring error-tolerance in SRAM architecture on hardware accelerated neural network

Kwon, Sangheon; Lee, Kyung-Min; Kim, Yoonsoo; Kim, Kyungah; Lee, Changmin; Ro, Won Woo

doi:10.1109/icce-asia.2016.7804818

Cited by 9 publications

(3 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Austin et al [7] discovered a correlation between weight error and classification accuracy for CNN and MLP. Kwon et al [8] introduced Gaussian noise to the weight parameters of LeNet and observed varying error tolerances among convolutional layers. However, the means of error injection and analysis methods for neural networks in these studies are relatively simple and didn't analyze the effect of SEU on neural networks by building models.…”

Section: Introductionmentioning

confidence: 99%

Quantifying the Impact of Single-Event Upsets on Neural Networks with Uncertainty Analysis

Yan,

Gao

2024

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

The Convolutional neural networks have gradually penetrated into various fields of social life, in addition to aerospace devices, neural network chips are also indispensable for target recognition and detection, etc. Due to the complex environment in space, neural network chips are extremely easy to be damaged by cosmic rays, so it is urgent to evaluate and enhance the robustness of neural network chips. However, most of the existing researches are monolithic error injection of neural networks and do not build relevant models for quantitative analysis. The goal of this paper is to analyse the effects of single-event upsets (SEUs) induced by the penetration of SRAM storage areas in a quantized neural network chip by energetic particles from space. Firstly, a SEU probability model as well as an uncertainty assessment model are established, and the effect of SEU on the reliability of CNNs is evaluated by fault injection with the upset rate of four chip types. The results of the error injection experiments show that the Virtex-5 type of rollover rate decreases the average accuracy of VGG-16 by 20%, while the other three types of error injection have an impact of about 11% on the accuracy of the neural network. Quantitative experimental results confirm the hypothesis that SEU increases the uncertainty of neural networks. The quantification of these effects contributes to the study of reinforcement of neural network chips.

show abstract

Section: Introductionmentioning

confidence: 99%

Quantifying the Impact of Single-Event Upsets on Neural Networks with Uncertainty Analysis

Yan,

Gao

2024

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

show abstract

“…Since not all the errors are critical for a CNN, it is crucial to accurately identify the critical layers of the CNN to balance the utilization of the TMR hardware resources and the hardening effectiveness. In [16], they tested the robustness of the CNN architecture via injecting Gaussian noise into the weights of the CNN and concluded that the error tolerance of a layer tended to worsen as it moved toward the output layer. However, the sensitivity of weight-related SEUs in different layers is influenced by various factors such as the number of weights, the weight value distribution, the network structure, and the layer position.…”

Section: Introductionmentioning

confidence: 99%

Evaluation and Mitigation of Weight-Related Single Event Upsets in a Convolutional Neural Network

Cai,

et al. 2024

Electronics

View full text Add to dashboard Cite

Single Event Upsets (SEUs) are most likely to cause bit flips within the trained parameters of a convolutional neural network (CNN). Therefore, it is crucial to analyze and implement hardening techniques to enhance their reliability under radiation. In this paper, random fault injections into the weights of LeNet-5 were carried out in order to evaluate and propose strategies to improve the reliability of a CNN. According to the results of an SEU fault injection, the accuracy of the CNN can be classified into the following three categories: benign conditions, poor conditions, and critical conditions. Two efficient methods for mitigating weight-related SEUs are proposed, as follows: weight limiting and Triple Modular Redundancy (TMR) for the critical bit of the critical layer. The hardening results show that when the number of SEU faults is small, the weight limiting almost completely eliminates the critical and poor conditions of LeNet-5’s accuracy. Additionally, even when the number of SEU faults is large enough, combining the weight limiting and TMR methods for the critical bit of the critical layer can retain the occurrence rate of benign conditions at 98%, saving 99.3% of the hardware resources compared to the Full-TMR hardening method.

show abstract

“…For example, [22] has evaluated the reliability of a CNN with 4 convolutional layers with bit flips on weights and showed that CNNs with large kernels are more robust than those with smaller kernels. In [23], LeNet-5 has been evaluated with Gaussian white noise on the weights, and it has been found that errors on the layers closer to the output layer have a larger effect on the network performance. The effects of errors on weights for different layers in LeNet-5 have been studied in [24]; convolution layers are more error resilient than full connection layers.…”

Section: I Introductionmentioning

confidence: 99%

On the Dependability of Bidirectional Encoder Representations from Transformers (BERT) to Soft Errors

Gao,

Su,

Wang

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

Measuring error-tolerance in SRAM architecture on hardware accelerated neural network

Cited by 9 publications

References 4 publications

Quantifying the Impact of Single-Event Upsets on Neural Networks with Uncertainty Analysis

Quantifying the Impact of Single-Event Upsets on Neural Networks with Uncertainty Analysis

Evaluation and Mitigation of Weight-Related Single Event Upsets in a Convolutional Neural Network

On the Dependability of Bidirectional Encoder Representations from Transformers (BERT) to Soft Errors

Contact Info

Product

Resources

About