Building a Polyhedral Representation from an Instrumented Execution

Selva, Manuel; Gruber, Fabian; Sampaio, Diogo; Guillon, Christophe; Pouchet, Louis-Noël; Rastello, Fabrice

doi:10.1145/3363785

Cited by 4 publications

(1 citation statement)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Polyhedral analysis, while providing an excellent foundation to rigorously reason the legality of, and explore the space of, loop transformations, can be an "overkill" to capture the relatively simple data structures (tensors) and operations (without loop-carried dependencies) in DNNs. Moreover, polyhedral analysis is normally limited to affine-loop analysis and transformations (although latest efforts [61,71,72] do extend it to certain non-affine loop optimizations), and cannot capture certain operation (combinations) in DNNs. An example will be a combination of Gather, which copies input to output indirectly using an index array followed by Flatten, which changes the dimensionality of a tensor.…”

Section: Introductionmentioning

confidence: 99%

DNNFusion: accelerating deep neural networks execution with advanced operator fusion

Niu

Guan

Wang

et al. 2021

Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation

View full text Add to dashboard Cite

Deep Neural Networks (DNNs) have emerged as the core enabler of many major applications on mobile devices. To achieve high accuracy, DNN models have become increasingly deep with hundreds or even thousands of operator layers, leading to high memory and computational requirements for inference. Operator fusion (or kernel/layer fusion) is key optimization in many state-of-the-art DNN execution frameworks, such as TensorFlow, TVM, and MNN, that aim to improve the efficiency of the DNN inference. However, these frameworks usually adopt fusion approaches based on certain patterns that are too restrictive to cover the diversity of operators and layer connections, especially those seen in many extremely deep models. Polyhedral-based loop fusion techniques, on the other hand, work on a low-level view of the computation without operator-level information, and can also miss potential fusion opportunities. To address this challenge, this paper proposes a novel and extensive loop fusion framework called DNNFusion. The basic idea of this work is to work at an operator view of DNNs, but expand fusion opportunities by developing a classification of both individual operators and their combinations. In addition, DNNFusion includes 1) a novel mathematical-propertybased graph rewriting framework to reduce evaluation costs and facilitate subsequent operator fusion, 2) an integrated fusion plan generation that leverages the high-level analysis and accurate light-weight profiling, and 3) additional optimizations during fusion code generation. DNNFusion is extensively evaluated on 15 DNN models with varied types

show abstract

Section: Introductionmentioning

confidence: 99%