SmartShuttle: Optimizing off-chip memory accesses for deep learning accelerators

Communication Lower Bound in Convolution Accelerators

“…Inputs, weights, and outputs are pruned in [19]. Our work targets at general CNN accelerators without data pruning/compression, so the results reported in [10], [19] are not directly comparable to ours. Instead, we try to make an approximate comparison.…”

“…Rather than using a single dataflow, several studies have integrated multiple dataflows into an accelerator (with increased hardware cost) and selected the best one according to the layer dimensions [19]- [23]. These approaches usually perform better than the approaches based on a single dataflow.…”

“…In the aforementioned approaches, some studies ( [7], [11], [19], [26]- [29]) have considered communication optimization, while the others mainly focus on the computational components. Besides the three categories, there are other communication optimization approaches for CNN accelerators (e.g., the fused-layer approach [35] that optimizes data movement between convolutional layers).…”

“…Ref. [19] considered an OutR dataflow but the tiling sizes were not properly selected. The convolution implementation proposed in [41] is for graphics processing units rather than for hardware accelerators.…”

“…Data reuse heavily depends on the convolutional dataflow. There are various approaches to optimize the dataflow: 1) designing an elaborate dataflow [5], [7]- [18], 2) selecting the best dataflow from several candidates [19]- [23], and 3) design space exploration (DSE) [24]- [32]. A fair number of these studies focus on the performance and/or the energy efficiency of the computational components.…”