Soft Error Tolerant Convolutional Neural Networks on FPGAs With Ensemble Learning

Gao, Zhen; Zhang, Han; Yao, Yi; Reviriego, Pedro; Zeng, Shulin; Ge, Guangjun; Wang, Yu; Ullah, Anees

doi:10.1109/tvlsi.2021.3138491

Cited by 20 publications

(7 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[245]-[249] Redundancy-based [77], [85], [91], [101], [104], [250]- [256] Fault masking [91], [92], [257], [258], [259], [260], [261] Variation-aware mapping for memristor crossbar arrays [262], [263] ECC [81], [83], [84], [264] ML-based [265] Adaptive training after testing [146], [266] Razor [257], [267], [268] Neuron adaptation [269] Aging-aware on-line training of memristor crossbar arrays [270],…”

Section: Model-basedmentioning

confidence: 99%

See 1 more Smart Citation

Testability and Dependability of AI Hardware: Survey, Trends, Challenges, and Perspectives

Liu

Stratigopoulos³

2023

IEEE Des. Test

View full text Add to dashboard Cite

In recent years, there has been an expedited trend in embracing bold and radical innovation of computer architectures, aiming at the continuation of computing performance improvement despite the slowed-down physical device scaling. One new frontier in this field focuses on Artificial Intelligence (AI) hardware. While functionality of AI hardware still remains the main focus, testability and dependability of these new architectures need to be addressed before the mainstream adoption. This survey paper covers the state-of-the-art in research and development of dependability and testability solutions for AI hardware including digital or analog implementations of Artificial Neural Networks (ANNs) and Spiking Neural Networks (SNNs), used in accelerators and neuromorphic designs. Trends, challenges and perspectives are also discussed in this paper.

show abstract

Section: Model-basedmentioning

confidence: 99%

“…Finally, a redundancy-based fault-tolerance strategy based on ensemble learning is proposed in [256]. Ensemble learning consists of training a set of independent smaller and weak (i.e, with lower base networks, using different network structures, learning algorithms, and training datasets.…”

Section: Proactive Hardware-based Approachesmentioning

confidence: 99%

Testability and Dependability of AI Hardware: Survey, Trends, Challenges, and Perspectives

Liu

Stratigopoulos³

2023

IEEE Des. Test

View full text Add to dashboard Cite

show abstract

“…In addition, there are also conventional fault-tolerant computing mechanisms closely combined with neural network accelerator architectures. Zhen Gao et al [30] introduced ensemble learning to fault-tolerant neural network processing for the first time and combined it with redundancy design. The basic idea is to have a group of redundant base neural network models implemented in parallel and equipped the different implementations with a score comparison voter on FPGAs such that the majority of the soft error induced prediction errors can be mitigated with negligible hardware overhead.…”

Section: A Related Workmentioning

confidence: 99%

“…Model layer fault-tolerant design approaches typically explore the inherent fault tolerance and redundancy in neural network models and have the models desensitized to computing variations and input variations induced by various hardware faults. Architecture layer fault-tolerant approaches mitigate hardware faults with specialized neural network accelerator architectures such as online recomputing [17] and runtime voting [30]. Circuit layer fault-tolerant approaches focus on low-level circuit designs such as error-tolerant encoding, fine-grained modular redundancy [31], and stochastic circuits [32].…”

Section: Introductionmentioning

confidence: 99%

Fault-Tolerant Deep Learning: A Hierarchical Perspective

Liu¹,

Gao²,

Liu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

With the rapid advancements of deep learning in the past decade, it can be foreseen that deep learning will be continuously deployed in more and more safety-critical applications such as autonomous driving and robotics. In this context, reliability turns out to be critical to the deployment of deep learning in these applications and gradually becomes a firstclass citizen among the major design metrics like performance and energy efficiency. Nevertheless, the back-box deep learning models combined with the diverse underlying hardware faults make resilient deep learning extremely challenging. In this special session, we conduct a comprehensive survey of fault-tolerant deep learning design approaches with a hierarchical perspective and investigate these approaches from model layer, architecture layer, circuit layer, and cross layer respectively.

show abstract

“…The above studies were performed on the whole system or layers in the network, and the effect of errors on the different modules in the accelerator was not examined. The authors in [27] and [28] evaluate the reliability of processing elements for the convolutional layers in the FPGA implemented VGGNet and ResNets. The fault injection experimental results show that most of the errors on the configuration memory could be tolerated by the network itself, but there are still errors on 12%~13% of the critical bits that could dramatically degrade performance.…”

Section: Introductionmentioning

confidence: 99%

Systematic Reliability Evaluation of FPGA Implemented CNN Accelerators

Gao

Yao

et al. 2023

IEEE Trans. Device Mater. Relib.

Self Cite

View full text Add to dashboard Cite

Convolutional neural networks (CNN) have become essential for many scientific and industrial applications, such as image classification and pattern detection. Among the devices that can implement neural networks, SRAM based FPGAs are a popular option due to their excellent parallel computing capability and good flexibility. However, SRAM-FPGAs are susceptible to radiation effects, which limits its application on safety critical applications. In this paper, the reliability of an accelerator based on the advanced Instruction-Set Architecture is evaluated based on hardware fault injection experiments. Each main module of the accelerator is evaluated separately, and the impact of parallelism and model features on the accelerator reliability is also examined. The experimental results reveal some important conclusions in terms of general hardware reliability and also of the particular model reliability. First, over 99% of SEUs on the computation modules will cause accuracy loss, and the reliability improves for higher parallelism. Second, a large portion of SEUs on the data mover and the instruction scheduler will cause system corruptions due to abnormal interactions with the ARM or other modules. Third, nonlinear activation and pooling layers are effective in reducing the effect of SEUs on computation modules, so models that use these layers tend to be more robust. The results provide a deep understanding of the impact of errors on CNNs implemented on ISA based FPGA accelerators (e.g., the Xilinx DPU).

show abstract

Soft Error Tolerant Convolutional Neural Networks on FPGAs With Ensemble Learning

Cited by 20 publications

References 45 publications

Testability and Dependability of AI Hardware: Survey, Trends, Challenges, and Perspectives

Testability and Dependability of AI Hardware: Survey, Trends, Challenges, and Perspectives

Fault-Tolerant Deep Learning: A Hierarchical Perspective

Systematic Reliability Evaluation of FPGA Implemented CNN Accelerators

Contact Info

Product

Resources

About