Ultra Power-Efficient CNN Domain Specific Accelerator with 9.3TOPS/Watt for Mobile and Embedded Applications

Abstract

Computer vision performances have been significantly improved in recent years by Convolutional Neural Networks (CNN). Currently, applications using CNN algorithms are deployed mainly on general purpose hardwares, such as CPUs, GPUs or FPGAs. However, power consumption, speed, accuracy, memory footprint, and die size should all be taken into consideration for mobile and embedded applications. Domain Specific Architecture (DSA) for CNN is the efficient and practical solution for CNN deployment and implementation. We designed and produced a 28nm Two-Dimensional CNN-DSA accelerator with an ultra powerefficient performance of 9.3TOPS/Watt and with all processing done in the internal memory instead of outside DRAM. It classifies 224x224 RGB image inputs at more than 140fps with peak power consumption at less than 300mW and an accuracy comparable to the VGG benchmark. The CNN-DSA accelerator is reconfigurable to support CNN model coefficients of various layer sizes and layer types, including convolution, depth-wise convolution, short-cut connections, max pooling, and ReLU. Furthermore, in order to better support real-world deployment for various application scenarios, especially with low-end mobile and embedded platforms and MCUs (Microcontroller Units), we also designed algorithms to fully utilize the CNN-DSA accelerator efficiently by reducing the dependency on external accelerator computation resources, including implementation of Fully-Connected (FC) layers within the accelerator and compression of extracted features from the CNN-DSA accelerator. Live demos with our CNN-DSA accelerator on mobile and embedded systems show its capabilities to be widely and practically applied in the real world.

CNN Inference Using a Preprocessing Precision Controller and Approximate Multipliers With Various Precisions

“…This section will present how a reconfigurable approximate multiplier with two precisions can be implemented to allow for the utilization of the concept in single-core systems. This will expand the proposed concept to include the design of approximate multiplier-based CNN accelerators for embedded systems and low power applications such as [42][43]. Fig 13. demonstrates how a reconfigurable approximate multiplier with two precisions can be implemented inside each MAC unit.…”

GnetSeg: Semantic Segmentation Model Optimized on a 224mW CNN Accelerator Chip at the Speed of 318FPS

“…Low power consumption and high inference speed are practical requirements when deploying semantic segmentation with constraints on power and computation resources. The recent years development and wide availability of lowpower CNN (Convolutional Neural Networks) accelerators [11,7] has made the semantic segmentation applications on mobile devices more feasible. There are some significant advantages of these CNN accelerator chips.…”

“…First, ultra-low power. The CNN accelerator chip in [11] has the low power consumption of only 300mW. The most recent released chip has the peak power of only 224mW.…”

GnetDet: Object Detection Optimized on a 224mW CNN Accelerator Chip at the Speed of 106FPS

“…It has many real-world applications to be deployed on edge devices. CNN (Convolutional Neural Networks) accelerators [19,15] are ideal for these applications by providing high inference speed and low power consumption. The most recent chip has the peak power of only 224mW [14].…”