OmniDRL: A 29.3 TFLOPS/W Deep Reinforcement Learning Processor with Dualmode Weight Compression and On-chip Sparse Weight Transposer

Lee, Juhyoung; Kim, Sangjin; Jo, Wooyoung; Han, Donghyeon; Lee, Jinsu; Yoo, Hoi‐Jun

doi:10.23919/vlsicircuits52068.2021.9492504

Cited by 14 publications

(21 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For more evaluation, we compare LearningGroup against the state-ofthe-art DNN training accelerators optimized for processing sparse data, such as EagerPruning [21], Procrustes [29], SparseTrain [22], and OmniDRL [35]. Since each accelerator has a different number of processing units and operating frequency, we compare how well each accelerator exploits the sparsity by measuring the speedup achieved in sparse data over the dense case.…”

Section: Comparison With Sparse Training Acceleratorsmentioning

confidence: 99%

“…Deep Reinforcement Learning Accelerator There have been an increasing number of works on accelerating deep reinforcement learning while they only focus on single-agent domain [35]- [38]. Kim et al [37] compress the input activations using the top three frequently used exponent values rather than pruning the weights.…”

Section: Related Workmentioning

confidence: 99%

“…Kim et al [37] compress the input activations using the top three frequently used exponent values rather than pruning the weights. OmniDRL [35] also uses weight pruning, but its pruning algorithm degrades the reward when applied to MARL. LearningGroup is the first work that targets online multi-agent reinforcement learning by utilizing a hardware-amenable pruning algorithm.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

LearningGroup: A Real-Time Sparse Training on FPGA via Learnable Weight Grouping for Multi-Agent Reinforcement Learning

Yang¹,

Kim²,

Kim³

2022

Preprint

View full text Add to dashboard Cite

Multi-agent reinforcement learning (MARL) is a powerful technology to construct interactive artificial intelligent systems in various applications such as multi-robot control and self-driving cars. Unlike supervised model or single-agent reinforcement learning, which actively exploits network pruning, it is obscure that how pruning will work in multi-agent reinforcement learning with its cooperative and interactive characteristics.In this paper, we present a real-time sparse training acceleration system named LearningGroup, which adopts network pruning on the training of MARL for the first time with an algorithm/architecture co-design approach. We create sparsity using a weight grouping algorithm and propose on-chip sparse data encoding loop (OSEL) that enables fast encoding with efficient implementation. Based on the OSEL's encoding format, LearningGroup performs efficient weight compression and computation workload allocation to multiple cores, where each core handles multiple sparse rows of the weight matrix simultaneously with vector processing units. As a result, Learn-ingGroup system minimizes the cycle time and memory footprint for sparse data generation up to 5.72× and 6.81×. Its FPGA accelerator shows 257.40-3629.48 GFLOPS throughput and 7.10-100.12 GFLOPS/W energy efficiency for various conditions in MARL, which are 7.13× higher and 12.43× more energy efficient than Nvidia Titan RTX GPU, thanks to the fully on-chip training and highly optimized dataflow/data format provided by FPGA. Most importantly, the accelerator shows speedup up to 12.52× for processing sparse data over the dense case, which is the highest among state-of-the-art sparse training accelerators.

show abstract

Section: Comparison With Sparse Training Acceleratorsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

LearningGroup: A Real-Time Sparse Training on FPGA via Learnable Weight Grouping for Multi-Agent Reinforcement Learning

Yang¹,

Kim²,

Kim³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Another paper [30] added a weight transpose-reading unit instead of modifying the main PE array. This additional unit generated the transposed weights before they were fetched to • Back-propagation: the PE array.…”

Section: B Architecture-level Solutionmentioning

confidence: 99%

“…Weight transposer suggested by [30] adopted hierarchical transpose-read and reduced memory access by excluding the fetch of pruned weights. Even though the weight transposer unit showed fast weight decoding speed, it can rather induce low area efficiency due to the additional large register file array.…”

Section: B Architecture-level Solutionmentioning

confidence: 99%

Energy-Efficient DNN Training Processors on Micro-AI Systems

Han

Kang

Kim

et al. 2022

IEEE Open J. Solid-State Circuits Soc.

View full text Add to dashboard Cite

Many edge/mobile devices are now able to utilize deep neural networks (DNNs) thanks to the development of mobile DNN accelerators. Mobile DNN accelerators overcame the problems of limited computing resources and battery capacity by realizing energy-efficient inference. However, its passive behavior makes it difficult for DNN to provide active customization for individual users or its service environment. The importance of onchip training is rising more and more to provide active interaction between DNN processors and ever-changing surroundings or conditions. Despite its advantages, the DNN training has more constraints than the inference such that it was considered impractical to be realized on mobile/edge devices. Recently, there are many trials to realize mobile DNN training, and a number of prior works will be summarized. Firstly, it arranges the new challenges of the DNN accelerator induced by training functionality and discusses new hardware features related to the challenges. Secondly, it explains algorithm-hardware cooptimization methods and explains why it becomes mainstream in mobile DNN training research. Thirdly, it compares the main differences between the conventional inference accelerators and recent training processors. Finally, the conclusion is made by proposing the future directions of the DNN training processor in micro-AI systems.

show abstract

An Overview of Energy-Efficient DNN Training Processors

Han

Yoo

2023

On-Chip Training NPU - Algorithm, Architecture and SoC Design

View full text Add to dashboard Cite

OmniDRL: A 29.3 TFLOPS/W Deep Reinforcement Learning Processor with Dualmode Weight Compression and On-chip Sparse Weight Transposer

Cited by 14 publications

References 1 publication

LearningGroup: A Real-Time Sparse Training on FPGA via Learnable Weight Grouping for Multi-Agent Reinforcement Learning

LearningGroup: A Real-Time Sparse Training on FPGA via Learnable Weight Grouping for Multi-Agent Reinforcement Learning

Energy-Efficient DNN Training Processors on Micro-AI Systems

An Overview of Energy-Efficient DNN Training Processors

Contact Info

Product

Resources

About