A Lightweight Error-Resiliency Mechanism for Deep Neural Networks

Goldstein, Brunno F.; Ferreira, Victor C.; Srinivasan, Sudarshan; Das, Dipankar; Nery, Alexandre S.; Kundu, Sandip; França, Felipe M. G.

doi:10.1109/isqed51717.2021.9424287

Cited by 12 publications

(9 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Code-based Error Detection: Code-based techniques have been used for the protection of data processing [27], [28], [29] and memory [30], [31] in DNNs. Feinberg et.…”

Section: Symptom-based Error Detection (Sed)mentioning

confidence: 99%

“…al. [27] employed a similar code-based scheme, i.e., AN-codes, to protect MAC PEs of a systolic array-based accelerator. They necessitated a specific data quantization during network training to reduce protection overhead.…”

Section: Symptom-based Error Detection (Sed)mentioning

confidence: 99%

“…Code-based techniques add information redundancy with the property that certain errors can be detected or sometimes corrected. While non-arithmetic codes, like Hamming and Cyclic codes, are well-suited for memory [30] and communication [35] protection, data processing can efficiently be protected through arithmetic codes like ANcodes and residue codes [27], [28], [36], [37].…”

Section: Arithmetic Error Codesmentioning

confidence: 99%

See 2 more Smart Citations

Analysis and Enhancement of Resilience for LSTM Accelerators Using Residue-Based CEDs

Nosrati,

Navabi

2024

IEEE Access

View full text Add to dashboard Cite

As Long Short-Term Memory (LSTM) accelerators are increasingly being employed in safetycritical applications with high-reliability demands, protecting them against errors becomes imperative. Traditional protection techniques for LSTMs are either costly, conflict with strict area and power constraints for neural network accelerators, or introduce performance overhead that is untenable for real-time and latency-critical accelerators. In this paper, we propose residue-based Concurrent Error Detection (CED) schemes to detect transient faults and alleviate their impact during LSTM computations. CED units are employed in a coarse-grain or fine-grain fashion, depending on the granularity of the components being protected from the whole LSTM computations to individual processing elements. In pursuit of a more costeffective strategy, we use fine-grain CEDs in a selective manner based on the resilience characteristics of LSTM. The selections are made spatially for LSTM synaptic weights or temporally based on LSTM time steps. The experimental results show that the proposed residue-based CEDs (i) can achieve nearly complete fault coverage even under extremely large bit error rates, (ii) significantly decrease misprediction rates compared to the unprotected LSTM, and (iii) incur low overhead without compromising performance. Our method is compared with modular redundancy techniques such as DMR and TMR (Double and Triple Modular Redundancy).

show abstract

“…Code-based Error Detection: Code-based techniques have been used for the protection of data processing [27], [28], [29] and memory [30], [31] in DNNs. Feinberg et.…”

Section: Symptom-based Error Detection (Sed)mentioning

confidence: 99%

Section: Symptom-based Error Detection (Sed)mentioning

confidence: 99%

Section: Arithmetic Error Codesmentioning

confidence: 99%

See 1 more Smart Citation

Analysis and Enhancement of Resilience for LSTM Accelerators Using Residue-Based CEDs

Nosrati,

Navabi

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…In [164], an AN code based fault detection mechanism is proposed to protect the MAC units of the DNN accelerator. AN codes add redundancy in the data to detect faults during arithmetic operations.…”

Section: On-line Test 1) Atpg and Functional Testingmentioning

confidence: 99%

Testability and Dependability of AI Hardware: Survey, Trends, Challenges, and Perspectives

Liu

Stratigopoulos³

2023

IEEE Des. Test

View full text Add to dashboard Cite

In recent years, there has been an expedited trend in embracing bold and radical innovation of computer architectures, aiming at the continuation of computing performance improvement despite the slowed-down physical device scaling. One new frontier in this field focuses on Artificial Intelligence (AI) hardware. While functionality of AI hardware still remains the main focus, testability and dependability of these new architectures need to be addressed before the mainstream adoption. This survey paper covers the state-of-the-art in research and development of dependability and testability solutions for AI hardware including digital or analog implementations of Artificial Neural Networks (ANNs) and Spiking Neural Networks (SNNs), used in accelerators and neuromorphic designs. Trends, challenges and perspectives are also discussed in this paper.

show abstract

“…They have been used in various safety-critical applications to increase the reliability of the systems. In [10] authors have used a specific class of arithmetic codes, known as AN-Codes, in state-of-the-art DNN accelerators. They have exhibited that they can achieve 99 % fault coverage with a 5-bit arithmetic code with minimal area and power overhead.…”

Section: Arithmetic Error Codesmentioning

confidence: 99%

A Survey on Fault-Tolerant Methodologies for Deep Neural Networks

Syed¹,

Ulbricht²,

Piotrowski³

et al. 2023

PAR

View full text Add to dashboard Cite

Asignificant rise in Artificial Intelligence (AI) has impacted many applications around us, so much so that AI has now been increasingly used in safety-critical applications. AI at the edge is the reality, which means performing the data computation closer to the source of the data, as opposed to performing it on the cloud. Safety-critical applications have strict reliability requirements; therefore, it is essential that AI models running on the edge (i.e., hardware) must fulfill the required safety standards. In the vast field of AI, Deep Neural Networks (DNNs) are the focal point of this survey as it has continued to produce extraordinary outcomes in various applications i.e. medical, automotive, aerospace, defense, etc. Traditional reliability techniques for DNNs implementation are not always practical, as they fail to exploit the unique characteristics of the DNNs. Furthermore, it is also essential to understand the targeted edge hardware because the impact of the faults can be different in ASICs and FPGAs. Therefore, in this survey, first, we have examined the impact of the fault in ASICs and FPGAs, and then we seek to provide a glimpse of the recent progress made towards the fault-tolerant DNNs. We have discussed several factors that can impact the reliability of the DNNs. Further, we have extended this discussion to shed light on many state-of-the-art fault mitigation techniques for DNNs.

show abstract

A Lightweight Error-Resiliency Mechanism for Deep Neural Networks

Cited by 12 publications

References 21 publications

Analysis and Enhancement of Resilience for LSTM Accelerators Using Residue-Based CEDs

Analysis and Enhancement of Resilience for LSTM Accelerators Using Residue-Based CEDs

Testability and Dependability of AI Hardware: Survey, Trends, Challenges, and Perspectives

A Survey on Fault-Tolerant Methodologies for Deep Neural Networks

Contact Info

Product

Resources

About