Accelerating the CNN Inference on FPGAs

Abstract

Convolutional Neural Networks (CNNs) are currently adopted to solve an ever greater number of problems, ranging from speech recognition to image classi cation and segmentation. The large amount of processing required by CNNs calls for dedicated and tailored hardware support methods. Moreover, CNN workloads have a streaming nature, well suited to recon gurable hardware architectures such as FPGAs.The amount and diversity of research on the subject of CNN FPGA acceleration within the last 3 years demonstrates the tremendous industrial and academic interest. This paper presents a state-of-the-art of CNN inference accelerators over FPGAs. The computational workloads, their parallelism and the involved memory accesses are analyzed. At the level of neurons, optimizations of the convolutional and fully connected layers are explained and the performances of the di erent methods compared. At the network level, approximate computing and datapath optimization methods are covered and state-of-the-art approaches compared. The methods and tools investigated in this survey represent the recent trends in FPGA CNN inference accelerators and will fuel the future advances on e cient hardware deep learning.

Radar Signal Processing for Sensing in Assisted Living: The Challenges Associated With Real-Time Implementation of Emerging Algorithms

“…GPUs are the most widely used platforms to implement CNNs due to their processing power (up to 11 TFLOP/s), FPGA is a real alternative for real-time radar data analysis in terms of power consumption (vs GPUs), rapid prototyping, and massively parallel computing capabilities at different data rates [28]. As a result, numerous FPGA-based CNN accelerators have been proposed, targeting both High Performance Computing data-centers and embedded applications [29,30].…”

“…Regarding real-time implementation, fixed-point arithmetic is privileged, limiting the performances and significantly decreasing the accuracy [39]. However, half-precision floating-point format seems to be interesting to address future implementations on FPGA as well as approximated computing to maintain good energy-performance trade-offs [30].…”

Low-Power Neural Networks for Semantic Segmentation of Satellite Images

“…Neural network pruning, on the other hand, is the process of removing some weights [18] or entire convolution filters [19,27] and their associated feature maps in a neural network, in order to extract a functional "sub-network" that has a lower computational complexity and similar accuracy. A lot of work has been done on enabling and accelerating NN inference on FPGA [1], with tools being released to automatically generate HDL code for any neural network architecture [10], with automated use of quantization and other simplification techniques. These approaches are orthogonal to our work as they could be applied to any neural network.…”

Hardware Acceleration of Computer Vision and Deep Learning Algorithms on the Edge using OpenCL

“…A custom hardware accelerator has challenges of its own, including cost of the hardware, as well as time-to-market for the acceleration solution [1] [2]. Field Programmable Gate Arrays (FPGAs) have proven to be reliable accelerators for rapidly changing industries.…”