A High-Performance FPGA-Based Depthwise Separable Convolution Accelerator

Abstract

Depthwise separable convolution (DSC) significantly reduces parameter and floating operations with an acceptable loss of accuracy and has been widely used in various lightweight convolutional neural network (CNN) models. In practical applications, however, DSC accelerators based on graphics processing units (GPUs) cannot fully exploit the performance of DSC and are unsuitable for mobile application scenarios. Moreover, low resource utilization due to idle engines is a common problem in DSC accelerator design. In this paper, a high-performance DSC hardware accelerator based on field-programmable gate arrays (FPGAs) is proposed. A highly reusable and scalable multiplication and accumulation engine is proposed to improve the utilization of computational resources. An efficient convolution algorithm is proposed for depthwise convolution (DWC) and pointwise convolution (PWC), respectively, to reduce the on-chip memory occupancy. Meanwhile, the proposed convolution algorithms achieve partial fusion between PWC and DWC, and improve the off-chip memory access efficiency. To maximise bandwidth utilization and reduce latency when reading feature maps, an address mapping method for off-chip accesses is proposed. The performance of the proposed accelerator is demonstrated by implementing MobileNetV2 on an Intel Arria 10 GX660 FPGA by using Verilog HDL. The experimental results show that the proposed DSC accelerator achieves a performance of 205.1 FPS, 128.8 GFLOPS, and 0.24 GOPS/DSP for input images of size 224×224×3.

Enhanced Autofocusing with a Compact and Swift ST-VGG Network

“…Similarly, Jang J G et al [34] found that depthwise separable convolutions, especially when combined with Falcon convolution, can decrease computational demands while preserving accuracy. Furthermore, Huang J et al [35] developed an FPGAbased DSC accelerator, enhancing on-chip and off-chip efficiency. These studies underline the value of depthwise separable convolutions in improving computational efficiency and model performance.…”

Efficient Two-Stage Max-Pooling Engines for an FPGA-Based Convolutional Neural Network

“…Among these options, FPGAs have gained popularity for implementing CNNs in embedded systems primarily due to their ability to perform convolution operations in parallel with high energy efficiency. Compared to GPUs, FPGAs offer higher energy efficiency, making them an attractive choice for resource-constrained embedded systems [4][5][6][7].…”