Fault Resilience Analysis of Quantized Deep Neural Networks

Syed, Rizwan Tariq; Ulbricht, Markus; Piotrowski, Krzysztof; Krstić, Miloš

doi:10.1109/miel52794.2021.9569094

Cited by 10 publications

(5 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For each BER, the fault injection simulations were performed at least 1000 times, and every simulation time was 24,550 clock cycles, which are the cycles needed by a complete classification task. Table 3 shows the corresponding error probabilities including the total number of serious errors and system crashes, and energy efficiency, which is defined by Equation (10).…”

Section: Comparisons Of Design Overhead and Fault Tolerancementioning

confidence: 99%

See 1 more Smart Citation

Implementation of Highly Reliable Convolutional Neural Network with Low Overhead on Field-Programmable Gate Array

Chen,

Xie,

Huo

et al. 2024

Electronics

View full text Add to dashboard Cite

Due to the advantages of parallel architecture and low power consumption, a field-programmable gate array (FPGA) is typically utilized as the hardware for convolutional neural network (CNN) accelerators. However, SRAM-based FPGA devices are extremely susceptible to single-event upsets (SEUs) induced by space radiation. In this paper, a fault tolerance analysis and fault injection experiments are applied to a CNN accelerator, and the overall results show that SEUs occurring in a control unit (CTRL) lead to the highest system error rate, which is over 70%. After that, a hybrid hardening strategy consisting of a finite state machine error-correcting circuit (FSM-ECC) and a triple modular redundancy automatic hardening technique (TMR-AHT) is proposed in this paper to achieve a tradeoff between radiation reliability and design overhead. Moreover, the proposed methodology has very small workload and good migration ability. Finally, by full exploiting the fault tolerance property of CNNs, a highly reliable CNN accelerator with the proposed hybrid hardening strategy is implemented with Xilinx Zynq-7035. When BER is 2 × 10−6, the proposed hybrid hardening strategy reduces the whole system error rate by 78.95% with the overhead of an extra 20.7% of look-up tables (LUTs) and 20.9% of flip-flops (FFs).

show abstract

Section: Comparisons Of Design Overhead and Fault Tolerancementioning

confidence: 99%

“…Refs. [10,16] utilized Quant, SMART, TMR, and SMART+TMR, respectively, to harden the MLP, and the error probability was decreased by 39.96%, 43.84%, 53.97%, and 55.52%, respectively.…”

Section: Comparisons Of Design Overhead and Fault Tolerancementioning

confidence: 99%

Implementation of Highly Reliable Convolutional Neural Network with Low Overhead on Field-Programmable Gate Array

Chen,

Xie,

Huo

et al. 2024

Electronics

View full text Add to dashboard Cite

show abstract

“…For instance, consider a weight parameter value of a DNN model represented as a floating-point 32 (FP32) number. A fault in the most significant exponent bit of the FP32 number can substantially change the value of the DNN's parameter and dramatically decrease the accuracy [36]. The majority of studies have only considered fault injection in the weights of the neural network.…”

Section: ) Fault Injection At Rtl Levelmentioning

confidence: 99%

FPGA Implementation of a Fault-Tolerant Fused and Branched CNN Accelerator With Reconfigurable Capabilities

Syed,

Zhao,

Chen

et al. 2024

IEEE Access

View full text Add to dashboard Cite

The ImageNet moment was a turning point for Convolutional Neural Networks (CNNs), as it demonstrated their potential to revolutionize computer vision tasks. This triumph of CNNs has motivated solving even more complex problems involving multiple tasks from multiple data modalities. Conventionally, a single CNN accelerator has been optimized to perform just one task or multiple correlated tasks. This study presents a shared-layers approach that leverages the pattern-learning capabilities of CNNs to perform multiple uncorrelated tasks from different modalities using a single hardware accelerator. We overcame the challenge of data imbalance in multi-modal learning by synthetic data generation. We achieved an average classification accuracy above 90% on a single CNN accelerator, which would otherwise require three accelerators. Due to the reliability concerns imposed by transistor shrinking and aging, we extended the shared layers methodology and introduced a fault-tolerant CNN accelerator with reconfigurable capabilities supporting fault-tolerant (FT), high-performance (HP), and de-stress (DS) modes. FT mode provides high reliability against soft errors utilizing double/triple modular redundancy, HP mode offers peak performance of 0.979 TOPs using parallel execution, and DS mode reduces dynamic power consumption by up to 68.6% in clock-gated design and even more using a partial reconfiguration method, contributing to decelerating the aging process of the circuit. We have comprehensively evaluated two different CNN architectures (i.e., fused and branched), for three distinct tasks, in three different operating modes, based on accuracy, quantization, pruning, hardware resource utilization, power, energy, performance, and reliability.

show abstract

“…If not masked, this fault could propagate through the DNN network and drastically decrease the accuracy. The impact of a fault also causes a deviation in FxP numbers, which leads to a decrease in accuracy, but the overall impact would be less due to the less dynamic range of the FxP numbers [39]. Therefore it is crucial to define a data type and bit-width, which can fulfill the requirement of accuracy and reliability, and hardware resources.…”

Section: Factors Affecting the Resiliency Of Deep Neural Networkmentioning

confidence: 99%

A Survey on Fault-Tolerant Methodologies for Deep Neural Networks

Syed¹,

Ulbricht²,

Piotrowski³

et al. 2023

PAR

View full text Add to dashboard Cite

Asignificant rise in Artificial Intelligence (AI) has impacted many applications around us, so much so that AI has now been increasingly used in safety-critical applications. AI at the edge is the reality, which means performing the data computation closer to the source of the data, as opposed to performing it on the cloud. Safety-critical applications have strict reliability requirements; therefore, it is essential that AI models running on the edge (i.e., hardware) must fulfill the required safety standards. In the vast field of AI, Deep Neural Networks (DNNs) are the focal point of this survey as it has continued to produce extraordinary outcomes in various applications i.e. medical, automotive, aerospace, defense, etc. Traditional reliability techniques for DNNs implementation are not always practical, as they fail to exploit the unique characteristics of the DNNs. Furthermore, it is also essential to understand the targeted edge hardware because the impact of the faults can be different in ASICs and FPGAs. Therefore, in this survey, first, we have examined the impact of the fault in ASICs and FPGAs, and then we seek to provide a glimpse of the recent progress made towards the fault-tolerant DNNs. We have discussed several factors that can impact the reliability of the DNNs. Further, we have extended this discussion to shed light on many state-of-the-art fault mitigation techniques for DNNs.

show abstract

Fault Resilience Analysis of Quantized Deep Neural Networks

Cited by 10 publications

References 5 publications

Implementation of Highly Reliable Convolutional Neural Network with Low Overhead on Field-Programmable Gate Array

Implementation of Highly Reliable Convolutional Neural Network with Low Overhead on Field-Programmable Gate Array

FPGA Implementation of a Fault-Tolerant Fused and Branched CNN Accelerator With Reconfigurable Capabilities

A Survey on Fault-Tolerant Methodologies for Deep Neural Networks

Contact Info

Product

Resources

About