DiBA: n-Dimensional Bitslice Architecture for LSTM Implementation

Roodsari, Mahboobe Sadeghipour; Saber, Mohamad Ali; Navabi, Zainalabedin

doi:10.1109/ddecs50862.2020.9095614

Cited by 5 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Limitations: Our residue-based CEDs are designed for networks with fixed-point data types. Although fixed-point data types are prevalent in the inference phase [6], [7], [23], [25], [26], [28], some systems may use floating-point data types [2]. Such systems cannot be protected with our CEDs, because of complexities in floating-point arithmetic.…”

Section: Discussionmentioning

confidence: 99%

“…hidden state (6) where wjx and wjh (j=i,f,o,c) are input and hidden weight matrices and bj (j=i,f,o,c) represents the biases. ct is memory data at time step t. ⊙ denotes element-wise multiplication.…”

Section: Background a Long Short-term Memory (Lstm) Networkmentioning

confidence: 99%

“…To conduct these evaluations, we implemented the RTL design of the proposed CEDs in a synthesizable VHDL format and incorporated them into the RTL VHDL description of an LSTM accelerator. For the LSTM accelerator, we considered the n-dimensional DiBA architecture [6] and proceeded to map the LSTM networks onto it. The hardware accelerator and the proposed protection mechanisms are synthesized with the 45 nm technology node using the Nangate open cell library, operating at a frequency of 200 MHz with a supply voltage of 1.1 V.…”

Section: Run Time Costmentioning

confidence: 99%

“…Table III investigates the impact of LSTM accelerator architecture and the number of PEs on the area overhead of the proposed methods. In this investigation, we explore both 1-D architectures, such as those employed by DiBA [6] and LSTM-Sharp [2], and 2-D architectures that use the systolic array style [2]. To facilitate a meaningful comparison, for a 2-D architecture, we examine a typical systolic array with identical PE structures as DiBA.…”

Section: Run Time Costmentioning

confidence: 99%

“…For efficient performance of LSTMs in the inference phase, dedicated hardware accelerators are employed [2], [6], [7]. However, these accelerators are susceptible to faults stemming from fabrication technology and hardware architectures [8], [9], [10], [11], [12].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Analysis and Enhancement of Resilience for LSTM Accelerators Using Residue-Based CEDs

Nosrati,

Navabi

2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

As Long Short-Term Memory (LSTM) accelerators are increasingly being employed in safetycritical applications with high-reliability demands, protecting them against errors becomes imperative. Traditional protection techniques for LSTMs are either costly, conflict with strict area and power constraints for neural network accelerators, or introduce performance overhead that is untenable for real-time and latency-critical accelerators. In this paper, we propose residue-based Concurrent Error Detection (CED) schemes to detect transient faults and alleviate their impact during LSTM computations. CED units are employed in a coarse-grain or fine-grain fashion, depending on the granularity of the components being protected from the whole LSTM computations to individual processing elements. In pursuit of a more costeffective strategy, we use fine-grain CEDs in a selective manner based on the resilience characteristics of LSTM. The selections are made spatially for LSTM synaptic weights or temporally based on LSTM time steps. The experimental results show that the proposed residue-based CEDs (i) can achieve nearly complete fault coverage even under extremely large bit error rates, (ii) significantly decrease misprediction rates compared to the unprotected LSTM, and (iii) incur low overhead without compromising performance. Our method is compared with modular redundancy techniques such as DMR and TMR (Double and Triple Modular Redundancy).

show abstract