Roofline-Model-Based Design Space Exploration for Dataflow Techniques of CNN Accelerators

Abstract

To effectively compute convolutional layers, a complex design space must exist (e.g., the dataflow techniques associated with the layer parameters, loop transformation techniques, and hardware parameters). For efficient design space exploration (DSE) of various dataflow techniques, namely, the weight-stationary (WS), output-stationary (OS), row-stationary (RS), and no local reuse (NLR) techniques, the processing element (PE) structure and computational pattern of each dataflow technique are analyzed. Various performance metrics are calculated, namely, the throughput (in giga-operations per second, GOPS), computation-to-communication ratio (CCR), on-chip memory usage, and off-chip memory bandwidth, as closed-form expressions of the layer and hardware parameters. In addition, loop interchange and loop unrolling techniques with a double-buffer architecture are assumed. Many roofline model-based simulations are performed to explore relevant dataflow techniques for a wide variety of convolutional layers of typical neural networks. Through simulation, this paper provides insights into the trends in accelerator performance as the layer parameters change. For convolutional layers with large input and output feature map (ifmap and ofmap) widths and heights, the GOPS of the NLR dataflow technique tends to be higher than that of the techniques. For convolutional layers with low weight and ofmap widths and heights, the RS dataflow technique achieves optimal GOPS and on-chip memory usage. In the case of convolutional layers with small weight widths and heights, the GOPS of the WS dataflow technique tends to be high. In the case of convolutional layers with small ofmap widths and heights, the OS dataflow technique achieves optimal GOPS and on-chip memory usage. INDEX TERMS Accelerator, convolutional neural networks (CNNs), dataflow techniques, roofline, simulation, processing element (PE), design space exploration (DSE), field-programmable gate array (FPGA)

Multiply-and-Fire (MNF): An Event-driven Sparse Neural Network Accelerator

“…These results are estimated using Timeloop[30] and Accelergy[37] tools. Our data show that our event-driven technique consumes less energy than alternate data flows such as weight, output, and input stationary[32,35]. The specifications of three different layers studied are described in Table1.…”

“…However, these deep networks [14,21,25,33,36] can have multiple hidden layers, millions of parameters, billions of operations and require tremendous storage and intense computation resources, making it difficult to realize energy-efficient and high-performance solutions. To address this issue, several model compression techniques [10,17,18,20,26,38], efficient dataflow techniques [32,35], and dataflow accelerators [1,3,5,7,8,15,16,19,28,31,39,40,41,42,43] have been proposed and widely investigated in recent years. * These authors contributed equally to this work.…”

“…Several sparse CNN (s-CNN) accelerators have recently been proposed to exploit the structured [28,43] as well as the unstructured sparsity [1,3,5,7,8,15,16,19,31,39,40,41,42] in both CNN model parameters and its activations. Although this works [1,3,5,7,8,15,16,19,28,31,39,40,41,42,43] outperform dense CNN accelerators in terms of energy efficiency and performance, they use compressed vector-vector multiplication (VVM) or matrix-vector multiplication (MVM) engines based on traditional dataflow techniques such as weight stationary, input stationary, output stationary, and row stationary techniques [32,35]. When the sparsity of inputs or weights becomes excessively large (≥ 60%-90%), previous VVM and MVM accelerators suffer from low utilization issues.…”

An efficient loop tiling framework for convolutional neural network inference accelerators

“…In 2016, Motamedi et al adopted the design space exploration method to explore the optimal design parameters [19]. In this study, the roofline model [11] is to explore the best performance on a certain hardware platform, and the optimal values of Tn, Tm, Tr, and Tc are selected. The roofline model is presented in Figure 9.…”

“…Lu et al [10] introduced an efficient sparse Winograd method to accelerate CNN on FPGA. Besides, the roofline model is adopted to explore the efficient calculation of convolutional layer [11], and performance modelling is designed for CNN inference accelerators [12].…”