EDD: Efficient Differentiable DNN Architecture and Implementation Co-search for Embedded AI Solutions

Abstract

High quality AI solutions require joint optimization of AI algorithms and their hardware implementations. In this work, we are the first to propose a fully simultaneous, Efficient Differentiable DNN (deep neural network) architecture and implementation cosearch (EDD) methodology. We formulate the co-search problem by fusing DNN search variables and hardware implementation variables into one solution space, and maximize both algorithm accuracy and hardware implementation quality. The formulation is differentiable with respect to the fused variables, so that gradient descent algorithm can be applied to greatly reduce the search time. The formulation is also applicable for various devices with different objectives. In the experiments, we demonstrate the effectiveness of our EDD methodology by searching for three representative DNNs, targeting low-latency GPU implementation and FPGA implementations with both recursive and pipelined architectures. Each model produced by EDD achieves similar accuracy as the best existing DNN models searched by neural architecture search (NAS) methods on ImageNet, but with superior performance obtained within 12 GPU-hour searches. Our DNN targeting GPU is 1.40× faster than the state-of-the-art solution reported in Proxyless [1], and our DNN targeting FPGA delivers 1.45× higher throughput than the state-of-the-art solution reported in DNNBuilder [2].

Algorithm and Hardware Co-design for Reconfigurable CNN Accelerator

“…Nevertheless, a common drawback in these RL-based algorithm-hardware codesign approaches is that they demand significant number of GPU hours for search of both algorithm and hardware-defining parameters, which is unfeasible for real-life applications. To reduce the search cost, differentiable NAS (DNA) has been used in algorithm and hardware co-design [13]. However, it has been demonstrated in [16] that the NNs found by DNA can only achieve similar accuracy to the NNs generated by random search.…”

“…1) Hardware Design Space: This paper adopts an example design which uses a single configurable processing unit to process different layers. Although there are other designs, such as the streaming design [13], [15] with layer-wise reconfigurability, they usually require a large amount of on-chip memory to cache all the intermediate results, which restricts the model size of the NN and limits the neural architecture space. In this paper, we adopted the single processing engine design, such that our search space encompasses larger CNNs.…”

“…We compared our proposed approach with four other state-of-the-art co-design methods, including Co-Explore [11], EDD [13], HAO [26], and OFA [17]. Although there are other co-design works, they suffer from low accuracy [9], [10], [15], [30] or only evaluated on a small dataset [12], [14], [31].…”

“…Given its reconfigurability, FPGA can be utilized to provide highly-optimized hardware, customized for different NNs found by NAS. Previous work has attempted to apply evolutionary algorithm [8]- [10], reinforcement learning [11], [12] and differentiable NAS [13], [14] on algorithmhardware co-design for NNs on FPGA. However, these approaches iteratively perform the network training and design space exploration for multiple times, making the process timeconsuming.…”

RT-RCG: Neural Network and Accelerator Search Towards Effective and Real-time ECG Reconstruction from Intracardiac Electrograms

“…DNN Algorithm and Accelerator Co-exploration. Exploring the networks and the corresponding accelerators in a joint manner [1,31,39,40,50,92] has shown great potential towards maximizing both accuracy and efficiency. Recent works have extended NAS to jointly search DNN accelerators in addition to DNN structures.…”