Deep learning algorithms have seen success in a wide variety of applications, such as machine translation, image and speech recognition, and self-driving cars. However, these algorithms have only recently gained a foothold in the embedded systems domain. Most embedded systems are based on cheap microcontrollers with limited memory capacity, and, thus, are typically seen as not capable of running deep learning algorithms. Nevertheless, we consider that advancements in compression of neural networks and neural network architecture, coupled with an optimized instruction set architecture, could make microcontroller-grade processors suitable for specific low-intensity deep learning applications. We propose a simple instruction set extension with two main components-hardware loops and dot product instructions. To evaluate the effectiveness of the extension, we developed optimized assembly functions for the fully connected and convolutional neural network layers. When using the extensions and the optimized assembly functions, we achieve an average clock cycle count decrease of 73% for a small scale convolutional neural network. On a per layer base, our optimizations decrease the clock cycle count for fully connected layers and convolutional layers by 72% and 78%, respectively. The average energy consumption per inference decreases by 73%. We have shown that adding just hardware loops and dot product instructions has a significant positive effect on processor efficiency in computing neural network functions.INDEX TERMS Deep learning, embedded systems, instruction set optimization, RISC-V.

No abstract

We present a configurable implementation of a convolution processing unit suitable for computing mixed-precision quantized neural networks. We used Chisel to write the hardware generator, it is a framework for writing hardware circuit generators. Our generator is designed to use minimal hardware resources and is flexible in various aspects of the convolution operation, including: image size, kernel size, image bitwidth, kernel bitwidth, activation function, and more. The processing unit is static after generation, thus we don't pay the price of using more general hardware, instead we can tailor it to the problem at hand.