AutoDNNchip

Abstract

Recent breakthroughs in Deep Neural Networks (DNNs) have fueled a growing demand for domain-specific hardware accelerators (i.e., DNN chips). However, designing DNN chips is non-trivial because: (1) mainstream DNNs have millions of parameters and operations; (2) the design space is large due to the numerous design choices of dataflows, processing elements, memory hierarchy, etc.; and (3) an algorithm/hardware co-design is needed to allow the same DNN functionality to have a different decomposition, which would require different hardware IPs that correspond to dramatically different performance/energy/area tradeoffs. Therefore, DNN chips often take months to years to design and require a large team of cross-disciplinary experts. To enable fast and effective DNN chip design, we propose AutoDNNchip − a DNN chip generator that can automatically generate both FPGA-and ASIC-based DNN chip implementation (i.e., synthesizable RTL code with optimized algorithm-to-hardware mapping (i.e., dataflow) ) given DNNs from machine learning frameworks (e.g., PyTorch) for a designated application and dataset without humans in the loop. Specifically, AutoDNNchip consists of two integrated enablers: (1) a Chip Predictor, built on top of a graph-based accelerator representation, which can accurately and efficiently predict a DNN accelerator's energy, throughput, latency, and area based on the DNN model parameters, hardware configuration, technology-based IPs, and platform constraints; and (2) a Chip Builder, which can automatically explore the design space of DNN chips (including IP selection, block configuration, resource balance, etc.), optimize chip design via the Chip Predictor, and then generate synthesizable RTL code with optimized dataflows to achieve the target design metrics. Experimental results show that our Chip Predictor's predicted performance differs from real-measured ones by <10% when validated using 15 DNN models and 4 platforms (edge-FPGA/TPU/GPU and ASIC). Furthermore, both the FPGA-and ASIC-based DNN accelerators generated by our AutoDNNchip can achieve better (up to 3.86× improvement) performance than that of expert-crafted state-of-the-art accelerators, showing the effectiveness of AutoDNNchip. Our open-source code can be found at https://github.com/RICE-EIC/AutoDNNchip.git.

Special Session: Towards an Agile Design Methodology for Efficient, Reliable, and Secure ML Systems

“…The definitions specify simulation functionality, execution costs like area, latency, or energy [46], or hardware synthesis of each component [43]. Frequently used high-level components can also be defined, such as vector units, systolic array, and computation/memory tiles [47], as specified in libraries like MAGNet [1] and AutoDNNChip [48]. Library also integrates reliability and security costs for components and specialized components such as razor flip flops for detecting timing violations or trusted memory.…”

Physically Accurate Learning-based Performance Prediction of Hardware-accelerated ML Algorithms

“…Timeloop [3], MAESTRO [4], and DeepOpt [5] propose DNN dataflow analysis frameworks for inference accelerators, but their evaluations focus only on convolution layers, and the work in [6] proposes an energy estimation model for the convolution operation only. While the modeling efforts in [7], [8] include pooling and tensor addition along with convolution, their scope is limited to inference and do not have support for training operations. TRIM [9] proposes a design space explorer for DNN training, but does not support batch normalization, a heavy training workload, and is not evaluated on mainstream ASIC accelerators [1], [10]- [13] that use systolic or vector dot-product style hardware.…”

DeepBurning-GL

“…However, because the conventional design flow of RTL programming is complicated and error-prone, it takes considerable design efforts to realize the customized GNN accelerator case by case for the target GNN workload or on the target FPGA devices. Though there is a lot of prior study on automating the flow of DNN accelerator development for FPGA [5,24,31], it is non-trivial to directly apply prior DNN-FPGA automation frameworks to the GNN-based applications. The reason is mainly attributed to three factors including 1) they are built on the traditional deep learning frameworks such as Caffe and Pytorch.…”