A survey On hardware accelerators and optimization techniques for RNNs

Mittal, Sparsh; Umesh, Sumanth

doi:10.1016/j.sysarc.2020.101839

Cited by 32 publications

(19 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After applying the CNN acceleration, the RNN became the new bottleneck to be targeted for acceleration. In existing works, RNNs are also candidates for acceleration on FPGA [29], and their accelerators present better energy efficiency than CPU/GPU [30]. Unlike these works, our challenge is to achieve an efficient implementation alongside the DPU and extend the HW/SW interface to integrate the RNN in the Vitis AI environment.…”

Section: Discussionmentioning

confidence: 99%

Multimodal Neural Network Acceleration on a Hybrid CPU-FPGA Architecture: A Case Study

Ajili

Hara-Azumi

2022

IEEE Access

View full text Add to dashboard Cite

Internet of Things and deep learning (DL) are merging into one domain and enabling outstanding technologies for various classification tasks. Such technologies are based on complex networks that mainly target powerful platforms with rich computing resources, such as servers. Therefore, for resource-constrained embedded systems, new challenges of size, performance (i.e., latency, throughput, and accuracy), and power consumption-aware networks should be addressed, particularly when edge devices handle multimodal data (i.e., different types of real-time sensing data). In this case study, we focus on DeepSense, a time-series multimodal DL framework combining convolutional neural networks (NN) and recurrent NN to process accelerometer and gyroscope data for human activity recognition. We present a field-programmable gate array (FPGA)-based acceleration for DeepSense incorporated into a hardware/software co-design approach to achieve better latency and energy efficiency using the Xilinx Vitis AI framework. The architecture of DeepSense has drawbacks that cannot be easily alleviated by Vitis AI; therefore, we introduced a new methodology of adjusting the framework and its components (i.e., the deep learning processing unit (DPU)) to achieve a custom design suitable for such time-series multimodal NN. We implemented the accelerator on two FPGA boards and performed a quantitative evaluation by varying the DPU parameter settings to support our design approach. We demonstrated the effectiveness of our implementation against the original software implementation on mobile devices by achieving up to 2.5× and 5.2× improvement in latency and energy consumption, respectively. Through this case study, we provide crucial insights into the FPGAbased accelerator design of multimodal NN and essential aspects to consider for further improvements and adaptation in other application domains.

show abstract

Section: Discussionmentioning

confidence: 99%

Multimodal Neural Network Acceleration on a Hybrid CPU-FPGA Architecture: A Case Study

Ajili

Hara-Azumi

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…This includes the training of different types of layers, such as CNN (Convolutional Neural Network) or RNN (Recurrent Neural Network) layers. An overview of CNN inference accelerators on FPGA can be found, for example, in [1], while a survey of accelerators for recurrent neural networks, including LSTMs, can be found in [50]. In addition to publications that focus on the acceleration of DNN inference, some publications tackle the problem of implementing backpropagation for neural network training on FPGAs as well.…”

Section: Neural Network In Fpga-based Drl Implementationsmentioning

confidence: 99%

A Survey of Domain-Specific Architectures for Reinforcement Learning

Rothmann

Porrmann

2022

IEEE Access

View full text Add to dashboard Cite

Reinforcement learning algorithms have been very successful at solving sequential decisionmaking problems in many different problem domains. However, their training is often time-consuming, with training times ranging from multiple hours to weeks. The development of domain-specific architectures for reinforcement learning promises faster computation times, decreased experiment turn-around time, and improved energy efficiency. This paper presents a review of hardware architectures for the acceleration of reinforcement learning algorithms. FPGA-based implementations are the focus of this work, but GPU-based approaches are considered as well. Both tabular and deep reinforcement learning algorithms are included in this survey. The techniques employed in different implementations are highlighted and compared. Finally, possible areas for future work are suggested, based on the preceding discussion of existing architectures.

show abstract

“…In hardware, over 70 implementations on ASIC, FPGA, or GPU/CPU have been demonstrated in Ref. [60], but only two were applied to action recognition using video footage [61], [62]. To our knowledge, no RNN for animal behavior monitoring on hardware has been described.…”

Section: Proposed Cow Behavior Distribution Estimating Neural Network...mentioning

confidence: 99%

TinyCowNet: Memory- and Power-Minimized RNNs Implementable on Tiny Edge Devices for Lifelong Cow Behavior Distribution Estimation

et al. 2022

View full text Add to dashboard Cite

A survey On hardware accelerators and optimization techniques for RNNs

Cited by 32 publications

References 28 publications

Multimodal Neural Network Acceleration on a Hybrid CPU-FPGA Architecture: A Case Study

Multimodal Neural Network Acceleration on a Hybrid CPU-FPGA Architecture: A Case Study

A Survey of Domain-Specific Architectures for Reinforcement Learning

TinyCowNet: Memory- and Power-Minimized RNNs Implementable on Tiny Edge Devices for Lifelong Cow Behavior Distribution Estimation

Contact Info

Product

Resources

About