A Uniform Architecture Design for Accelerating 2D and 3D CNNs on FPGAs

Abstract

Three-dimensional convolutional neural networks (3D CNNs) have gained popularity in many complicated computer vision applications. Many customized accelerators based on FPGAs are proposed for 2D CNNs, while very few are for 3D CNNs. Three-D CNNs are far more computationally intensive and the design space for 3D CNN acceleration has been further expanded since one more dimension is introduced, making it a big challenge to accelerate 3D CNNs on FPGAs. Motivated by the finding that the computation patterns of 2D and 3D CNNs are very similar, we propose a uniform architecture design for accelerating both 2D and 3D CNNs in this paper. The uniform architecture is based on the idea of mapping convolutions to matrix multiplications. A customized mapping module is developed to generate the feature matrix tilings with no need to store the entire enlarged feature matrix on-chip or off-chip, a splitting strategy is adopted to reconstruct a convolutional layer to adapt to the on-chip memory capacity, and a 2D multiply-and-accumulate (MAC) array is adopted to compute matrix multiplications efficiently. For demonstration, we implement an accelerator prototype with a high-level synthesis (HLS) methodology on a Xilinx VC709 board and test the accelerator on three typical CNN models: AlexNet, VGG16, and C3D. Experimental results show that the accelerator achieves state-of-the-art throughput performance on both 2D and 3D CNNs, with much better energy efficiency than the CPU and GPU.

The Improved Division-Less MT-Type Velocity Estimation Algorithm for Low-Cost FPGAs

“…It is obvious that the error dynamics in Equations (18) and 22are driven by acceleration, which is manifested by velocity change. If we define acceleration a k = ∆v a k /T s that is always bounded in practice such that |a k | ≤ a max , then by considering 0 ≤ δt k < T s , we can evaluate the r.h.s.…”

“…In the case of the sinusoidal variation of δt, we can observe relatively good tracking for both algorithms; however, a more smooth response with slightly better accuracy was achieved by the DLMT1 algorithm (as can be observed by comparison of Figure 8c,d. The tracking performance of the DLMT1 algorithm is better than the DLMT2 algorithm simply because the velocity error in the first case is affected by the difference ∆δt only (see Equation (38)), whereas in the second case, the δt may have a significant impact as well (see Equation (18)). In the case of the sawtooth variation of δt, we can observe high-velocity peaks when the sawtooth generates abrupt changes from zero to one.…”

“…FPGAs can also ensure highly accurate and short sampling periods, which are required for advanced motion control applications. A short processing time of control algorithms is also possible; however, the feature-rich FPGAs have overwhelming resources that are required for performing complex computation algorithms [17,18] yet are still expensive, which may prevent their further use in low-cost applications. One of the challenges in the electronics industry nowadays concerns fitting complex circuitry into small spaces.…”

Optimized Compression for Implementing Convolutional Neural Networks on FPGA

“…Meng develops a deep model, termed the Gabor CNN, to address the computing-resource-saving problem [13]. Liu proposes a uniform architecture design by mapping convolutions to matrix multiplications for accelerating both two dimensional (2D) and three dimensional (3D) CNNs [14]. Despite the progress made, those accelerators take the algorithm as a black box, only focusing on hardware architecture optimization.…”

High-Level Synthesis of Multiclass SVM Using Code Refactoring to Classify Brain Cancer from Hyperspectral Images

“…The current rise in the complexity of the applications and the increment of the capabilities of silicon technologies, as well as the so called time to market constrain, make HLS methodologies and tools of mandatory use in the near future [1]. Due to the multiple commercial solutions that can be found in the market for multiprocessor system-on-chips (MPSoCs) nowadays, it is strictly necessary to improve its techniques and methodologies [2] so that the technology is able to deal with the multiple implementation possibilities by using high-level design [3,4].…”