An FPGA‐based JPEG preprocessing accelerator for image classification

Abstract

The FPGA‐based image classification accelerator has achieved success in many practical applications. However, most accelerators focus on solving the problem of convolution computation efficiency. End‐to‐end image classification involves many non‐convolutional operations, which can also become performance bottlenecks. Therefore, the authors propose an FPGA‐based JPEG preprocessing accelerator, which can accelerate non‐convolution operations of JPEG before feature extraction. To improve throughput and energy efficiency, four hardware structures are adopted in the design: 1) adaptive image block; 2) fast IDCT; 3) image block buffer; and 4) image block self‐location. The proposed design is evaluated on Xilinx XCZU7EV. The authors compare it with the optimized implementation of CPU and GPU, and the energy efficiency is improved by 23.07 times and 4.21 times, respectively. The throughput is 2.52 times better than the CPU implementation. And the authors demonstrate its practicality through a case study of image classification. These experimental results demonstrate its superior performance in terms of throughput and energy efficiency.

A Review of the Optimal Design of Neural Networks Based on FPGA

“…As shown in Figure 6, according to different design concepts and requirements, FPGA-based neural network optimization technology can be roughly divided into optimization for data and operation, optimization for bandwidth, and optimization for memory and access, among others, which are introduced in detail below. [71][72][73][74][75][76][77][78], less computations [79][80][81], improve calculation speed [82][83][84][85], Winograd fast convolution algorithm [86][87][88][89][90][91], Im2col convolution optimization algorithm [92][93][94][95][96][97], pipelined design [98][99][100][101][102], Roofline model [103][104][105], ping-pong cache [106][107][108][109], input feature map reuse [110,111], filter reuse [111,112], convolutional reuse [110]…”

“…Aiming at the data itself, a method to reduce the data accuracy and computational complexity is usually used. For example, fixed-point Roof-line model [103][104][105], ping-pong cache [106][107][108][109], input feature map reuse [110,111], filter reuse [111,112], convolutional reuse [110][111][112], time reuse or space reuse [111], standardize data access and storage [113][114][115]).…”

“…It met the requirements of miniaturization, integration, real time, and intelligent development of a visual processing system. In 2022, Tian-Yang Li et al [109] studied the preprocessing of Joint Photographic Experts Group (JPEG) images; optimized the inverse discrete cosine transform (IDCT) algorithm, which is time-consuming in JPEG decoding; and used the ping-pong cache to store the transition matrix. The throughput of 875.67 FPS (frame per second) and energy efficiency of 0.014 J/F were achieved on a Xilinx XCZU7EV.…”

“…The application of MobileNetv2 on FPGA achieved a throughput of 413.2 GOPS. Different from the general direction of optimizing convolution computation, the authors of [109] focused on optimizing non-convolution operations in order to improve throughput and energy efficiency. Some scholars based their work on quantization technology in order to reduce the amount of neural network data, so as to accelerate the calculation of face direction recognition [149].…”

“…Although compared with and GPU, this architecture had 665 times and 24.9 times the performance speedup ratio, respectively, and 3180 times and 138 times the efficiency speedup ratio, respectively, this architecture only performs sparse data processing without optimizing the most resource-consuming and time-consuming multiplication operations, which greatly limits the performance of the accelerator. general direction of optimizing convolution computation, the authors of [109] focused on optimizing non-convolution operations in order to improve throughput and energy efficiency. Some scholars based their work on quantization technology in order to reduce the amount of neural network data, so as to accelerate the calculation of face direction recognition [149].…”