Toolflows for Mapping Convolutional Neural Networks on FPGAs

Abstract

In the past decade, Convolutional Neural Networks (CNNs) have demonstrated state-of-the-art performance in various Artificial Intelligence tasks. To accelerate the experimentation and development of CNNs, several software frameworks have been released, primarily targeting power-hungry CPUs and GPUs. In this context, reconfigurable hardware in the form of FPGAs constitutes a potential alternative platform that can be integrated in the existing deep learning ecosystem to provide a tunable balance between performance, power consumption and programmability. In this paper, a survey of the existing CNN-to-FPGA toolflows is presented, comprising a comparative study of their key characteristics which include the supported applications, architectural choices, design space exploration methods and achieved performance. Moreover, major challenges and objectives introduced by the latest trends in CNN algorithmic research are identified and presented. Finally, a uniform evaluation methodology is proposed, aiming at the comprehensive, complete and in-depth evaluation of CNN-to-FPGA toolflows. ACM Reference format: Stylianos I. Venieris, Alexandros Kouris and Christos-Savvas Bouganis. 2018. Toolflows for Mapping Convolutional Neural Networks on FPGAs: A Survey and Future Directions. ACM Comput. Surv. 0, 0, Article 0 (March 2018), 36 pages. https://doi.org/10.1145/nnnnnnn.nnnnnnn INTRODUCTIONConvolutional Neural Networks (CNNs) [47] have demonstrated remarkable performance in Artificial Intelligence (AI) tasks. Being able to achieve high accuracy and frequently outperform traditional AI approaches, CNNs have been employed in a vast range of applications over the last decade, from object detection [72][53] and classification [78][82] to drone navigation [20] and autonomous driving [11] [7]. While becoming the state-of-the-art algorithm in AI fields such as machine vision, CNNs are challenged to deal with tasks of continuously increasing complexity. This leads to the design of deeper, more expressive networks at the expense of an increase in computational and memory requirements.

fpgaConvNet: Mapping Regular and Irregular Convolutional Neural Networks on FPGAs

“…Moreover, [53] and [54] presented automated frameworks specifically tailored for FPGA-based binarised and spiking neural networks respectively, while [55] proposed a library for the mapping of ConvNets on diverse embedded platforms, together with a comparative study of their design spaces. Finally, [12] provides a detailed survey of ConvNet-to-FPGA toolflows.…”

“…Nevertheless, several issues increase the complexity of Con-vNet system development on FPGAs [12]. With FPGAs' size and resource specifications advancing at a fast pace and with ConvNets becoming more complex, the possible mappings of a ConvNet on an FPGA lie on a large multidimensional design space that cannot be explored manually.…”

Cascade^CNN: Pushing the Performance Limits of Quantisation in Convolutional Neural Networks

“…while the number of elements included in the unrolled sliding window for all channels of the input feature map volume is P = K H K W N IN (7). In the case of FC Layers, batching is employed to form a similar R F C ×P matrix, each row of which contains the input feature vector of size P for a different input sample, and hence: R FC = BatchSize (8).…”

“…In this context, FPGAs constitute a promising platform for CNN inference due to their customisability which enables the use of optimised low-precision arithmetic units to achieve performance gains [7]. Existing FPGA-based accelerators have produced hardware designs that span from uniform 16-bit precision [8] [9] with minimal effect on accuracy, down to very high-performance binarised networks [10], but at a significant accuracy loss.…”

Improving Performance Estimation for FPGA-Based Accelerators for Convolutional Neural Networks

“…The standard 2D convolution layers, from which the CNN is constructed, occupy over 90% of the overall processing time [17] and their latency T i on the accelerator needs to be estimated to determine the best hardware configuration through DSE. For 2D convolution, there are several categories of parallelism including filter parallelism (P F ) or channel parallelism (P C) in addition to spatial and kernel parallelisms.…”