DeFiNES: Enabling Fast Exploration of the Depth-first Scheduling Space for DNN Accelerators through Analytical Modeling

Mei, Linyan; Goetschalckx, Koen; Symons, Arne; Verhelst, Marian

doi:10.48550/arxiv.2212.05344

Cited by 1 publication

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1 in different colors/patterns, and their overlap caused by 3x3 convolutions is marked in purple/crossed. FFMT was first employed for reducing peak memory usage in [9], but their path discovery [32] RAM reduction -Full Distributed Inference [30] RAM reduction ROM reduction Partly Manual Tiling [5,9] RAM reduction -Automated Tiling [6,10,19,[23][24][25][26] requires partially manual user effort. Other works that use FFMT with automated path discovery are [5,10,19,25,26].…”

Section: Related Workmentioning

confidence: 99%

“…FFMT along with tiling in the depthwise dimension for single layers without operator fusion was explored in [6,23]. Based on these, the work in [24] identifies the full FFMT search space of loop scheduling in a memory hierarchy and adds a new cost model. [24] further states: "in most convolution layers all input channels are required to calculate any output feature, which makes cross-layer tiling across the channel dimensions impossible", which we challenge with this work.…”

Section: Related Workmentioning

confidence: 99%

“…Based on these, the work in [24] identifies the full FFMT search space of loop scheduling in a memory hierarchy and adds a new cost model. [24] further states: "in most convolution layers all input channels are required to calculate any output feature, which makes cross-layer tiling across the channel dimensions impossible", which we challenge with this work. Our distinction from existing work is summarized in Table 1.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Driver Generation for IoT Nodes With Optimization of the Hardware/Software Interface

Stahl

Mueller-Gritschneder

Schlichtmann

2020

IEEE Embedded Syst. Lett.

View full text Add to dashboard Cite

Memory optimization for deep neural network (DNN) inference gains high relevance with the emergence of TinyML, which refers to the deployment of DNN inference tasks on tiny, low-power microcontrollers. Applications such as audio keyword detection or radar-based gesture recognition are heavily constrained by the limited memory on such tiny devices because DNN inference requires large intermediate run-time buffers to store activations and other intermediate data, which leads to high memory usage. In this paper, we propose a new Fused Depthwise Tiling (FDT) method for the memory optimization of DNNs, which, compared to existing tiling methods, reduces memory usage without inducing any run time overhead. FDT applies to a larger variety of network layers than existing tiling methods that focus on convolutions. It improves TinyML memory optimization significantly by reducing memory of models where this was not possible before and additionally providing alternative design points for models that show high run time overhead with existing methods. In order to identify the best tiling configuration, an end-to-end flow with a new path discovery method is proposed, which applies FDT and existing tiling methods in a fully automated way, including the scheduling of the operations and planning of the layout of buffers in memory. Out of seven evaluated models, FDT achieved significant memory reduction for two models by 76.2% and 18.1% where existing tiling methods could not be applied. Two other models showed a significant run time overhead with existing methods and FDT provided alternative design points with no overhead but reduced memory savings.

show abstract