CoopNet: Cooperative Convolutional Neural Network for Low-Power MCUs

Abstract

Fixed-point quantization and binarization are two reduction methods adopted to deploy Convolutional Neural Networks (CNNs) on end-nodes powered by low-power microcontroller units (MCUs). While most of the existing works use them as stand-alone optimizations, this work aims at demonstrating there is margin for a joint cooperation that leads to inferential engines with lower latency and higher accuracy. Called CoopNet, the proposed heterogeneous model is conceived, implemented and tested on off-the-shelf MCUs with small on-chip memory and few computational resources. Experimental results conducted on three different CNNs using as test-bench the low-power RISC core of the Cortex-M family by ARM validate the CoopNet proposal by showing substantial improvements w.r.t. designs where quantization and binarization are applied separately.

TwinDNN: A Tale of Two Deep Neural Networks

“…Furthermore, our hardware and software design provides a true real-time parallel inference scheme, which allows its users to exploit all resources for the entire time. Such customization of accelerators and their concurrent execution are big advantages of FPGAs, and this is fundamentally different from the sequential approaches that previous studies [18,19,20] took, which could make one network idle when the other network is running.…”

“…However, in this study, both DSP and LUT utilizations are maximized in a flexible and efficient way by implementing two different networks with different main computation units. This is an aspect that CPU-based solutions [18] or even previous FPGA-based solutions [19,20] did not offer. Previous FPGA-based solutions either implemented only one of the networks on FPGAs [20], which resulted in LUT being a bottleneck, or did not use extremely low bit-width networks [19], which resulted in DSP being a bottleneck.…”

“…There have already been researches on this concept of hierarchical neural network design, where compressed and original networks are both used in neural network inference [18,19,20]. They have succeeded in achieving balanced accuracy and latency results by using different techniques, such as low-power MCU [18], or FPGA [19,20], to realize the concept. Although this work also mainly uses the concept of hierarchical neural network design, there are several major differences from previous studies.…”

Μnas

“…Initially, elements of gradient-based learning were combined with other machine learning algorithms [12,19] to produce ultra-low memory classifiers (< 2KB). More recently, manually designed neural networks with quantisation and binarisation have been used for image classification on MCUs, too, though with a larger memory footprint [26,37]. Neural architecture search (NAS) is a widely explored topic in deep learning for GPUs.…”

TentacleNet: A Pseudo-Ensemble Template for Accurate Binary Convolutional Neural Networks

“…There exist different works that proposed the use of ensemble methods for deep neural networks. Remarkable results are reported in [26], where the authors adopted a boosting strategy on image classification, but also in CoopNet [27] which combines multiple precision models to improve accuracy and inference latency. Even more interesting, the concept of ensemble learning can be found in the internal architecture of the most recent CNN models.…”