Structured Operations: Modular Design of Code Generators for Tensor Compilers

An End-to-End Programming Model for AI Engine Architectures

“…While the primary value of MLIR is its support for efficient (quick) construction of IRs modeling novel domains, an undeniable secondary value is the ability to use existing dialects, corresponding to established programming models, in combination with novel transformations tailored to problem-specific hardware configurations. For example, while there has been much research on the use of MLIR to lower DNNs to high-performance CPU and GPU platforms (Vasilache et al, 2022), such as data-center class devices and high-powered mobile devices (e.g., expensive mobile phones), there is a dearth of work on efficiently targeting low-power edge devices, such as micro-controllers and single-board computers. Yet those latter edge devices, while relatively underpowered, can be an attractive DNN deployment target in instances where power is a scarce commodity, such as IoT, AgTech, and urban infrastructure monitoring.…”

“…#map3 = affine_map<(d0, d1, d2) -> (d0, d2)> #map4 = affine_map<(d0, d1, d2) -> (d2, d1)> #map5 = affine_map<(d0, d1, d2) -> (d0, d1)> %3 = linalg.generic { indexing_maps = [#map3, #map4, #map5], iterator_types = ["parallel", "parallel", "reduction"] } ins(%arg0, %1 : tensor<32x32xf32>, tensor<32x32xf32>) outs(%2 : tensor<32x32xf32>) { ^bb0(%in: f32, %in_2: f32, %out: f32): %5 = arith.mulf %in, %in_2 : f32 %6 = arith.addf %out, %5 : f32 linalg.yield %6 : f32 } -> tensor<32x32xf32> opment time. In MLIR, the partial resolution to this problem is called structured code generation (Vasilache et al, 2022), i.e., high-level operations such as torch.nn. Linear(32,32) are first lowered to a structured representation, such as linalg.generic (see Listing 15), which is itself transformed and lowered to optimized loop-nests (see Listing 16).…”