Evaluating Fault Resiliency of Compressed Deep Neural Networks

Sabbagh, Majid; Gongye, Cheng; Fei, Yunsi; Wang, Yanzhi

doi:10.1109/icess.2019.8782505

Cited by 30 publications

(18 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When reporting the impact of bit-flips on the weights, we have adopted the Bit Error Rate with Zero Accuracy Drop (BERZAD) metric proposed by Sabbagh [2]. BERZAD is defined as the maximum bit-error rate (number of erroneous weight bits divided by the total number of weight bits) such that there is zero loss of accuracy, based on a 95% confidence interval.…”

Section: B Methodologymentioning

confidence: 99%

“…Many previous works have studied the fault tolerance of CNNs. Sabbagh [2] introduced the BERZAD metric and performed an analysis of LeNet-5 and VGG-16, which are no longer state of the art networks. In this work, no faultmitigation techniques are demonstrated.…”

Section: Related Workmentioning

confidence: 99%

“…Researchers have proposed hardware techniques, approaches based on identifying anomalies as well as techniques where the training process is modified. It is generally accepted that quantized neural networks are several orders of magnitude more robust than floating-point-based CNNs [2], [3]. The quantization process induces a loss of accuracy that may be problematic for some applications, therefore floating point formats remain important.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Zero-Overhead Protection for CNN Weights

Burel

Evans

Anghel

2021

2021 IEEE International Symposium on Defect and Fault Tolerance in VLSI and Nanotechnology Systems (DFT)

View full text Add to dashboard Cite

The numerical format used for representing weights and activations plays a key role in the computational efficiency and robustness of CNNs. Recently, a 16-bit floating point format called Brain-Float 16 (bf16) has been proposed and implemented in hardware accelerators. However, the robustness of accelerators implemented with this format has not yet been studied. In this paper, we perform a comparison of the robustness of state-of-the art CNNs implemented with 8-bit integer, Brain-Float 16 and 32bit floating point formats. We also introduce an error detection and masking technique, called opportunistic parity (OP), which can detect and mask errors in the weights with zero storage overhead. With this technique, the robustness of floating point weights to bit-flips can be improved by up to three orders of magnitude.

show abstract

Section: B Methodologymentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Zero-Overhead Protection for CNN Weights

Burel

Evans

Anghel

2021

2021 IEEE International Symposium on Defect and Fault Tolerance in VLSI and Nanotechnology Systems (DFT)

View full text Add to dashboard Cite

show abstract

“…Model weights are stored in the memory cells, which are susceptible to soft errors [17] and permanent faults [141]. The effect of hardware reliability issues on DNN processing has been simulated for CMOS devices [142] [143] and Resistive Random Access Memory (ReRAM) [126]. In contrast to the passive or deliberate aging defects, active malicious fault injection techniques such as laser beam [144] and rowhammer [145] can be utilized to modify the weight values stored in Static RAM (SRAM) and Dynamic RAM (DRAM).…”

Section: B Hardware-based Attacks On Deployed ML Modelmentioning

confidence: 99%

Two Sides of the Same Coin: Boons and Banes of Machine Learning in Hardware Security

Liu

Chang

Wang

et al. 2021

IEEE J. Emerg. Sel. Topics Circuits Syst.

View full text Add to dashboard Cite

The last decade has witnessed remarkable research advances at the intersection of machine learning (ML) and hardware security. The confluence of the two technologies has created many interesting and unique opportunities, but also left some issues in their wake. ML schemes have been extensively used to enhance the security and trust of embedded systems like hardware Trojans and malware detection. On the other hand, ML-based approaches have also been adopted by adversaries to assist side-channel attacks, reverse engineer integrated circuits and break hardware security primitives like Physically Unclonable Functions (PUFs). Deep learning is a subfield of ML. It can continuously learn from a large amount of labeled data with a layered structure. Despite the impressive outcomes demonstrated by deep learning in many application scenarios, the dark side of it has not been fully exposed yet. The inability to fully understand and explain what has been done within the super-intelligence can turn an inherently benevolent system into malevolent. Recent research has revealed that the outputs of Deep Neural Networks (DNNs) can be easily corrupted by imperceptibly small input perturbations. As computations are brought nearer to the source of data creation, the attack surface of DNN has also been extended from the input data to the edge devices. Accordingly, due to the opportunities of MLassisted security and the vulnerabilities of ML implementation, in this paper, we will survey the applications, vulnerabilities and fortification of ML from the perspective of hardware security. We will discuss the possible future research directions, and thereby, sharing a roadmap for the hardware security community in general.

show abstract

“…Vulnerabilities of DNN models to fault injections attacks are evaluated by different algorithms [23,39]. Our prior work [29] also considers the effect of model compression in fault resilience. A recent work implements practical fault attacks using laser beaming on a simple MLP inference engine running on a microcontroller [5].…”

Section: Side-channel Analysismentioning

confidence: 99%