Energy-Efficient DNN Inference on Approximate Accelerators Through Formal Property Exploration

Abstract

Deep Neural Networks (DNNs) are being heavily utilized in modern applications and are putting energy-constraint devices to the test. To bypass high energy consumption issues, approximate computing has been employed in DNN accelerators to balance out the accuracy-energy reduction trade-off. However, the approximation-induced accuracy loss can be very high and drastically degrade the performance of the DNN. Therefore, there is a need for a fine-grain mechanism that would assign specific DNN operations to approximation in order to maintain acceptable DNN accuracy, while also achieving low energy consumption. In this paper, we present an automated framework for weight-to-approximation mapping enabling formal property exploration for approximate DNN accelerators. At the MAC unit level, our experimental evaluation surpassed already energyefficient mappings by more than ×2 in terms of energy gains, while also supporting significantly more fine-grain control over the introduced approximation.

Dynamic Image Difficulty-Aware DNN Pruning

“…In the case of channel pruning, there are various methods for retraining the pruned model to recover the lost accuracy, such as fine-tuning the model on the pruned architecture or using knowledge distillation to transfer the knowledge from the original model to the pruned model [ 27 ]. However, these methods can be computationally expensive and time-consuming, and they may not be feasible or practical in all scenarios [ 18 ]. On the contrary, the proposed method does not involve retraining or fine-tuning; therefore, to keep evaluations fair, we avoid directly comparing our results with other pruning methods that do involve such techniques.…”

“…We combine the approaches presented in Section 2.1 and Section 2.2 to achieve lightweight DNN inference without compromises in terms of accuracy at run-time. We consider run-time scenarios where real-time performance is prioritized [ 18 ] and therefore target DNN inference where each image is processed independently rather than in big batches.…”