GnetSeg: Semantic Segmentation Model Optimized on a 224mW CNN Accelerator Chip at the Speed of 318FPS

Abstract

Semantic segmentation is the task to cluster pixels on an image belonging to the same class. It is widely used in the real-world applications including autonomous driving, medical imaging analysis, industrial inspection, smartphone camera for person segmentation and so on. Accelerating the semantic segmentation models on the mobile and edge devices are practical needs for the industry. Recent years have witnessed the wide availability of CNN (Convolutional Neural Networks) accelerators. They have the advantages on power efficiency, inference speed, which are ideal for accelerating the semantic segmentation models on the edge devices. However, the CNN accelerator chips also have the limitations on flexibility and memory. In addition, the CPU load is very critical because the CNN accelerator chip works as a co-processor with a host CPU. In this paper, we optimize the semantic segmentation model in order to fully utilize the limited memory and the supported operators on the CNN accelerator chips, and at the same time reduce the CPU load of the CNN model to zero. The resulting model is called GnetSeg. Furthermore, we propose the integer encoding for the mask of the GnetSeg model, which minimizes the latency of data transfer between the CNN accelerator and the host CPU. The experimental result shows that the model running on the 224mW chip achieves the speed of 318FPS with excellent accuracy for applications such as person segmentation.

GnetDet: Object Detection Optimized on a 224mW CNN Accelerator Chip at the Speed of 106FPS

“…Similar to the GnetSeg [14] model which is designed for semantic segmentation tasks on the CNN accelerator chip, the GnetDet model supports input image resolution of both 224x224 and 448x448, and supports RGB/YUV/Y format for the input channels.…”

“…CNN (Convolutional Neural Networks) accelerators [19,15] are ideal for these applications by providing high inference speed and low power consumption. The most recent chip has the peak power of only 224mW [14]. These low-power CNN accelerators are used in computer vision tasks [16,24,13], and also in NLP (Natural Language Processing) tasks [18,17,20,21,22,11], and extended into tabular data machine learning [25] and multimodal tasks [23].…”