Morphling: A Reconfigurable Architecture for Tensor Computation

Abstract

Tensor algebra plays a major role in various applications including data analysis, machine learning, and hydrodynamics simulation. Different tensor algebra inherently varies in dimension, size, and computation, leading to different execution preference, including parallelization, data arrangement, and accumulation. Another critical aspect for tensor algebra is the involved tensors can be with varying mixes of dense and sparse representation. Such diversified applications are notoriously difficult to accelerate. Prior ASIC architectures do not meet the needs due to fixed dataflow and prior fine-grained fabrics (e.g. FPGAs) solutions offer limited performance and power improvement due to bit-level reconfigurable structure.In this paper, we propose Morphling, a reconfigurable architecture that can flexibly handle both dense and sparse tensor computation. We first generalize a flexible execution model that decomposes tensor operations into three steps, including tensor vectorization, vector computation, and output reduction. The dense and sparse tensor computation share the same execution model, but differ in the vector computation step where the multiplications are conducted. Depending on the number of inputs and outputs that are linked together in the computation step, we define three parallel patterns including many-to-one, one-to-many, and one-to-one, which correspond to different implementation for dense and sparse computation. Furthermore, to efficiently support sparse tensor, we design a tiled-BCSR format that enables high parallelism and balanced workload. At architecture level, we propose a reconfigurable design to support the execution model. The hardware units can be reconfigured to support different datapath and enable different types of data reuse. We evaluate Morphling using various tensor operations and compare it with CPU, GPU, FPGA and state-of-the-art ASIC designs. Overall, Morphling achieves 13.4X, 677.7X, 44.7X energy efficiency over Xilinx ZC706 FPGA, Intel i7-9700K CPU, and NVIDIA TitanX GPU.

Automatic Generation of Spatial Accelerator for Tensor Algebra

“…NVDLA [41] and Flexflow [35] presented dataflow for DNN processing and Caffeine [56] presented a full-system evaluation. Morphling [34] and OMNI [31] designs a reconfigurable dataflow for both dense and sparse tensor computation, but it cannot support efficient spatial data reuse. [24] [15] presents efficient dataflows by using 3D-stacked memory in the accelerator.…”