Dynamically Reconfigurable Deep Learning for Efficient Video Processing in Smart IoT Systems

Eldash, Omar; Frost, Adam; Khalil, Kasem; Kumar, Ashok; Bayoumi, Magdy

doi:10.1109/wf-iot48130.2020.9221101

Cited by 6 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…x SEU [15] x yes (manufacturing defects) x x x permanent faults [16] x yes x x x permanent faults [3] bitstreams reconfiguration x yes x x x [17] x x yes yes x x [4] weights re-programming x x x x x [18] x x yes x yes x [37] voltage and frequency scaling x yes x x x [7] weights re-programming yes (radiation) yes x x SEU [5] large kernel decomposition x yes x x x [6] adaptive loading and processing data x yes x x x [8] hybrid quantization support x yes x x x [9] data path reconfiguration x yes x x x Proposed supporting multiple operating modes ( FT, DS, HP) yes(radiation, aging) yes yes yes SEU, SET, MBU, SEUs in CRAM forcing one application-specific CNN accelerator to learn the common features between the tasks, which would otherwise require three separate accelerators. We have evaluated fused and branched CNN models with different model compression methods.…”

Section: Reconfigurability Typementioning

confidence: 99%

“…In order to address the evolving challenges posed by changing AI application requirements, several research groups have proposed the idea of reconfigurable or adaptive accelerators. The central concept revolves around AI accelerators being Authors in [3] have different bitstream configurations for different DNN models with varying quantization levels for a video processing application. They dynamically change the bitstream of the DNN model for a tradeoff between accuracy and power.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

FPGA Implementation of a Fault-Tolerant Fused and Branched CNN Accelerator With Reconfigurable Capabilities

Syed,

Zhao,

Chen

et al. 2024

IEEE Access

View full text Add to dashboard Cite

The ImageNet moment was a turning point for Convolutional Neural Networks (CNNs), as it demonstrated their potential to revolutionize computer vision tasks. This triumph of CNNs has motivated solving even more complex problems involving multiple tasks from multiple data modalities. Conventionally, a single CNN accelerator has been optimized to perform just one task or multiple correlated tasks. This study presents a shared-layers approach that leverages the pattern-learning capabilities of CNNs to perform multiple uncorrelated tasks from different modalities using a single hardware accelerator. We overcame the challenge of data imbalance in multi-modal learning by synthetic data generation. We achieved an average classification accuracy above 90% on a single CNN accelerator, which would otherwise require three accelerators. Due to the reliability concerns imposed by transistor shrinking and aging, we extended the shared layers methodology and introduced a fault-tolerant CNN accelerator with reconfigurable capabilities supporting fault-tolerant (FT), high-performance (HP), and de-stress (DS) modes. FT mode provides high reliability against soft errors utilizing double/triple modular redundancy, HP mode offers peak performance of 0.979 TOPs using parallel execution, and DS mode reduces dynamic power consumption by up to 68.6% in clock-gated design and even more using a partial reconfiguration method, contributing to decelerating the aging process of the circuit. We have comprehensively evaluated two different CNN architectures (i.e., fused and branched), for three distinct tasks, in three different operating modes, based on accuracy, quantization, pruning, hardware resource utilization, power, energy, performance, and reliability.

show abstract