Algorithm-Hardware Co-Design of Adaptive Floating-Point Encodings for Resilient Deep Learning Inference

Tambe, Thierry; Yang, En-Yu; Wan, Zishen; Deng, Yuntian; Reddi, Vijay Janapa; Rushton, Gérard; Brooks, David; Wei, Gu-Yeon

doi:10.1109/dac18072.2020.9218516

Cited by 44 publications

(11 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By contrast, Q (1,4,11) is more robust and fits the model better, indicating that a data type which can optimally capture the parameter range can improve resilience. This is in line with some recent works [12], [15], [31].…”

Section: B Frl Drone Navigation Problem (Dronenav) 1) Problem Descrip...supporting

confidence: 93%

FRL-FI: Transient Fault Analysis for Federated Reinforcement Learning-Based Navigation Systems

Wan¹,

Anwar²,

Mahmoud³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Swarm intelligence is being increasingly deployed in autonomous systems, such as drones and unmanned vehicles. Federated reinforcement learning (FRL), a key swarm intelligence paradigm where agents interact with their own environments and cooperatively learn a consensus policy while preserving privacy, has recently shown potential advantages and gained popularity. However, transient faults are increasing in the hardware system with continuous technology node scaling and can pose threats to FRL systems. Meanwhile, conventional redundancy-based protection methods are challenging to deploy on resource-constrained edge applications. In this paper, we experimentally evaluate the fault tolerance of FRL navigation systems at various scales with respect to fault models, fault locations, learning algorithms, layer types, communication intervals, and data types at both training and inference stages. We further propose two cost-effective fault detection and recovery techniques that can achieve up to 3.3× improvement in resilience with <2.7% overhead in FRL systems.

show abstract

Section: B Frl Drone Navigation Problem (Dronenav) 1) Problem Descrip...supporting

confidence: 93%

FRL-FI: Transient Fault Analysis for Federated Reinforcement Learning-Based Navigation Systems

Wan¹,

Anwar²,

Mahmoud³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is noted that to achieve higher resiliency, data types should be able to optimally capture the value range rather than pursuing an unnecessarily large range. This is in line with some recent works [29][30][31].…”

Section: Inference In Drone Navigation Problemsupporting

confidence: 93%

Analyzing and Improving Fault Tolerance of Learning-Based Navigation Systems

Wan¹,

Anwar²,

Hsiao³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Learning-based navigation systems are widely used in autonomous applications, such as robotics, unmanned vehicles and drones. Specialized hardware accelerators have been proposed for highperformance and energy-efficiency for such navigational tasks. However, transient and permanent faults are increasing in hardware systems and can catastrophically violate tasks safety. Meanwhile, traditional redundancy-based protection methods are challenging to deploy on resource-constrained edge applications. In this paper, we experimentally evaluate the resilience of navigation systems with respect to algorithms, fault models and data types from both RL training and inference. We further propose two efficient fault mitigation techniques that achieve 2× success rate and 39% qualityof-flight improvement in learning-based navigation systems.

show abstract

“…Recent research proposals have described training of key deep learning models using even reduced precision floating point values (8-and 4-bit representations) [11,34,47,51]. Recently proposed AdaptiveFloat [48] is an inference-targeted floating-point format which maximizes its dynamic range at a network layer granularity by dynamically shifting its exponent range via modifications to the exponent bias and by optimally clipping (quantizing) its representable datapoints. Our proposed EFloat design practically achieves the same result without altering the exponent range and quantizing full-precision values.…”

Section: Related Workmentioning

confidence: 99%

EFloat: Entropy-coded Floating Point Format for Compressing Vector Embedding Models

Bordawekar¹,

Abali²,

Chen³

2021

Preprint

View full text Add to dashboard Cite

We describe the EFloat floating-point number format with 4 to 6 additional bits of precision and a wider exponent range than the existing floating point (FP) formats of any width including FP32, BFloat16, IEEE-Half precision, DLFloat, Ten-sorFloat, and 8-bit floats. In a large class of deep learning models we observe that FP exponent values tend to cluster around few unique values which presents entropy encoding opportunities. The EFloat format encodes frequent exponent values and signs with Huffman codes to minimize the average exponent field width. Saved bits then become available to the mantissa increasing the EFloat numeric precision on average by 4 to 6 bits compared to other FP formats of equal width. The proposed encoding concept may be beneficial to low-precision formats including 8-bit floats. Training deep learning models with low precision arithmetic is challenging. EFloat, with its increased precision may provide an opportunity for those tasks as well. We currently use the EFloat format for compressing and saving memory used in large NLP deep learning models. A potential hardware implementation for improving PCIe and memory bandwidth limitations of AI accelerators is also discussed.

show abstract

Algorithm-Hardware Co-Design of Adaptive Floating-Point Encodings for Resilient Deep Learning Inference

Cited by 44 publications

References 13 publications

FRL-FI: Transient Fault Analysis for Federated Reinforcement Learning-Based Navigation Systems

FRL-FI: Transient Fault Analysis for Federated Reinforcement Learning-Based Navigation Systems

Analyzing and Improving Fault Tolerance of Learning-Based Navigation Systems

EFloat: Entropy-coded Floating Point Format for Compressing Vector Embedding Models

Contact Info

Product

Resources

About