Design of an Energy-Efficient Accelerator for Training of Convolutional Neural Networks using Frequency-Domain Computation

Ko, Jong Hwan; Mudassar, Burhan Ahmad; Na, Taesik; Mukhopadhyay, Saibal

doi:10.1145/3061639.3062228

Cited by 54 publications

(31 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…There have been dense CNNs dataflows on FPGAs [6,16,21,43,44]. However, these dataflows will lead to invalid multiplications caused by the redundant connections between weights and input/ouput channels for sparse CNNs.…”

Section: B Dataflows For Dense Cnns On Fpgasmentioning

confidence: 99%

An Efficient Hardware Design for Accelerating Sparse CNNs With NAS-Based Models

Liang

Jin

et al. 2022

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

Deep convolutional neural networks (CNNs) have achieved remarkable performance at the cost of huge computation. As the CNN models become more complex and deeper, compressing CNNs to sparse by pruning the redundant connection in the networks has emerged as an attractive approach to reduce the amount of computation and memory requirement. On the other hand, FPGAs have been demonstrated to be an effective hardware platform to accelerate CNN inference. However, most existing FPGA accelerators focus on dense CNN models which are inefficient when executing sparse models as most of the arithmetic operations involve addition and multiplication with zero operands. In this work, we propose an accelerator with softwarehardware co-design for sparse CNNs on FPGAs. To efficiently deal with the irregular connections in the sparse convolutional layers, we propose a weight-oriented dataflow that exploits element-matrix multiplication as the key operation. Each weight is processed individually which yields low decoding overhead. Then we design an FPGA accelerator that features a tile lookup table (TLUT) and a channel multiplexer (CMUX). The tile look-up table is designed to match the index between sparse weights and input pixels. Using TLUT, the runtime decoding overhead is mitigated by using an efficient indexing operation. Moreover, we propose a weight layout to enable efficient on-chip memory access without conflicts. To cooperate with the weight layout, a channel multiplexer is inserted to locate the address. Last, we build a Neural Architecture Search (NAS) engine that leverages the reconfigurability of FPGAs to generate an efficient CNN model and choose the optimal hardware design parameters. Experiments demonstrate that our accelerator can achieve 223.4-309.0 GOP/s for the modern CNNs on Xilinx ZCU102, which provides a 2.4X-12.9X speedup over previous dense CNN accelerators on FPGAs. Our FPGA-aware NAS approach shows 2X speedup over MobileNetV2 with 1.5% accuracy loss.

show abstract

Section: B Dataflows For Dense Cnns On Fpgasmentioning

confidence: 99%

An Efficient Hardware Design for Accelerating Sparse CNNs With NAS-Based Models

Liang

Jin

et al. 2022

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

show abstract

“…但在文献 [34] 中, 并没有采用 Hermitian 对称的优化, 也没有使用三次乘法来算复数乘法的优化. 近年来, 也有在 FPGA 上采用 FFT 进行 CNN 训练的相关工作 [35] . 文献 [35] 的特点在于, 整个训练都在频域进行.…”

Section: 基于 Fft 的设计unclassified

“…近年来, 也有在 FPGA 上采用 FFT 进行 CNN 训练的相关工作 [35] . 文献 [35] 的特点在于, 整个训练都在频域进行. 传统的前向过程中, 每一层的输出都需要转换回时域再进行下一层的计算, 实际上这会导致很大的冗余变换操作.…”

Section: 基于 Fft 的设计unclassified

“…传统的前向过程中, 每一层的输出都需要转换回时域再进行下一层的计算, 实际上这会导致很大的冗余变换操作. 基于此, 文献 [35] 提出了频域的训练方式, 能够充分利用 FFT 算法卢丽强等: 面向卷积神经网络的 FPGA 设计表 2 非传统卷积算法在 FPGA 上的实现总结 [9,26,28,29,31,34,38,39] (2) 网络变换. 从表 3 [7, 9∼12, 14, 20, 21, 23, 24, 26, 39] 和 GoogleNet 采用了分支结构.…”

Section: 基于 Fft 的设计unclassified

See 1 more Smart Citation

Accelerating convolutional neural networks on FPGAs

Zheng

Xiao

et al. 2019

Sci. Sin.-Inf.

View full text Add to dashboard Cite

深度学习这一概念在 2006 年被人提出, 通过增加人工神经网络的深度增加算法提取数据特征的能力. 卷积神经网络 (convolutional neural network, CNN) 作为深度学习中被广泛采用的算法, 被人们实现在大量的应用中, 例如人脸识别、车道检测、立体视觉匹配、语音识别 [1∼4] , 如图 1 所示. 但是卷积神经网络的计算量庞大, 这就要求设计与其计算量相对应的加速器. 图形处理器 (graphic processing units, GPU)、现场可编程逻辑门阵列 (field-programmable gate array, FPGA)、专用集成电路 (application specific integrated circuit, ASIC) 是常用的神经网络加速器. 在这些加速器中, FPGA 因为它的高度并行计算、低功耗、可重复编程的特性而被人们广泛采用. 此外, 高层次综合 (high level synthesis, HLS) 为软件开发人员提供了良好的编程环境, 降低了 FPGA 的开发周期, 使得 FPGA 编程更加有效率.

show abstract

“…FFT-based CNN frameworks [27]- [30] are proposed for training networks with big kernels, but the advanced convnet frameworks use small kernels [31]- [33]. Jong Hwan Ko et al proposed an energy-efficient accelerator for the CNN using Fourier domain computations (abbreviated as koCNN) [34], in which the spectral pooling strategy [35] and the discrete sync interpolation operation are two important methods for Fourier domain training. Nevertheless, the costs for Fourier domain training are quite expensive.…”

Section: Introductionmentioning

confidence: 99%

A Fourier Domain Training Framework for Convolutional Neural Networks Based on the Fourier Domain Pyramid Pooling Method and Fourier Domain Exponential Linear Unit

Lin

Yao

2019

IEEE Access

View full text Add to dashboard Cite

Training convolutional neural networks (CNNs) in the frequency domain is of great significance for extending the deep learning principle to the frequency domain. However, the frequency domain representation of the convnet architecture is highly demanding due to their complicated Fourier domain training features. Therefore, high accuracy and unambiguous representation strategies are needed for training convolutional neural networks entirely in the Fourier domain. Being founded on the bin decomposition mechanism and the non-saturated activation theory, this paper proposes an accurate, stable and efficient Fourier domain training framework for convolutional neural networks. The framework contains two important Fourier domain representations: one is the Fourier domain exponential linear unit, and the other is the pyramid pooling layer. The former alleviates the vanishing phenomenon and makes CNNs easier to converge in the Fourier domain; the latter avoids the original cropping or warping steps and improves the classification accuracy. With the framework, the Fourier domain training accuracy is improved without sacrificing the throughput of the graphic processing unit (GPU). With the Re-50 as the backbone, the top-1 and top-5 classification errors are reduced from 28.85 and 9.55 to 18.63 and 4.05, respectively, while the speedup ratios of the framework can reach up to 4.9877 and 1.8997, respectively, at a batch size of 128 on an NVIDIA GEFORCE RTX 2080 GPU (8.92 TFLOPS). The average difference between the classification value and the ground truth value is only 0.21 on the MetaGram-1 set, which indicates great goodness-offit and robustness of the framework. This investigation illustrates that the proposed Fourier domain CNN framework using the sophisticated Fourier domain representation strategy is highly efficient and accurate. Therefore, it may serve as a baseline framework to establish the training pipelines for Fourier domain CNNs, which can improve the deep learning accuracy of CNNs and extend the Fourier domain representation strategy to other deep learning networks. INDEX TERMS Convolutional neural networks, deep learning, Fourier domain training, exponential linear unit, pyramid pooling.

show abstract

Design of an Energy-Efficient Accelerator for Training of Convolutional Neural Networks using Frequency-Domain Computation

Cited by 54 publications

References 3 publications

An Efficient Hardware Design for Accelerating Sparse CNNs With NAS-Based Models

An Efficient Hardware Design for Accelerating Sparse CNNs With NAS-Based Models

Accelerating convolutional neural networks on FPGAs

A Fourier Domain Training Framework for Convolutional Neural Networks Based on the Fourier Domain Pyramid Pooling Method and Fourier Domain Exponential Linear Unit

Contact Info

Product

Resources

About