Compilation of sparse array programming models

Henry, Rawn; Hsu, Olivia; Yadav, Rohan; Chou, Stephen Y.; Olukotun, Kunle; Amarasinghe, Saman; Kjølstad, Fredrik

doi:10.1145/3485505

Cited by 21 publications

(10 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Henry et al [35] generalized the sparse tensor algebra compilation theory from TACO [43] to support compilation of general dense and sparse array expressions, laying the foundation for a sparse NumPy-style system. In this system, arrays (tensors) can have any implicit fill value and any function can be computed across sparse and dense arrays.…”

Section: Sparse Compilersmentioning

confidence: 99%

Compiler Support for Sparse Tensor Computations in MLIR

Bik¹,

Koanantakool²,

Shpeisman³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Sparse tensors arise in problems in science, engineering, machine learning, and data analytics. Programs that operate on such tensors can exploit sparsity to reduce storage requirements and computational time. Developing and maintaining sparse software by hand, however, is a complex and error-prone task. Therefore, we propose treating sparsity as a property of tensors, not a tedious implementation task, and letting a sparse compiler generate sparse code automatically from a sparsity-agnostic definition of the computation. This paper discusses integrating this idea into MLIR.

show abstract

Section: Sparse Compilersmentioning

confidence: 99%

Compiler Support for Sparse Tensor Computations in MLIR

Bik¹,

Koanantakool²,

Shpeisman³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Our algorithm is summarized in Figure 4. A benefit of our algorithm to analyze complexity is that it will extend to alternate fill values and operators which are not + or •, extensions to the TACO compiler explored in [16]. Nothing in our algorithm is specific to the choice of 0 or the operators we have chosen in our examples.…”

Section: Automatic Asymptotic Analysismentioning

confidence: 99%

“…Kotlyar et al proposed implementing sparse tensor algebra using relational algebra and database techniques [21][22][23][24], but did not give any techniques to optimize the generated code. It is unclear if these techniques would extend to the additional formats and operations proposed in [11,16] operations into relational queries, and proposed a few techniques to optimize the resulting join structures. However, this technique was not generalized beyond matrices, nor does it apply directly to tensor compilers and the kinds of programs they generate [40].…”

Section: Related Workmentioning

confidence: 99%

“…As an example, consider TACO, a recently popular sparse tensor compiler that has inspired several lines of inquiry due to it's simplified abstractions for sparse compilation [17,18]. Such investigations include modifications to the compiler itself [11,12,16] or separate implementations such as COMET [36] or the MLIR sparse tensor dialect. TACO simplifies compilation of sparse tensor programs by considering each dimension separately.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Asymptotic Cost Model for Autoscheduling Sparse Tensor Programs

Ahrens¹,

Kjølstad²,

Amarasinghe³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

While loop reordering and fusion can make big impacts on the constant-factor performance of dense tensor programs, the effects on sparse tensor programs are asymptotic, often leading to orders of magnitude performance differences in practice. Sparse tensors also introduce a choice of compressed storage formats that can have asymptotic effects. Research into sparse tensor compilers has led to simplified languages that express these tradeoffs, but the user is expected to provide a schedule that makes the decisions. This is challenging because schedulers must anticipate the interaction between sparse formats, loop structure, potential sparsity patterns, and the compiler itself. Automating this decision making process stands to finally make sparse tensor compilers accessible to end users.We present, to the best of our knowledge, the first automatic asymptotic scheduler for sparse tensor programs. We provide an approach to abstractly represent the asymptotic cost of schedules and to choose between them. We narrow down the search space to a manageably small "Pareto frontier" of asymptotically undominated kernels. We test our approach by compiling these kernels with the TACO sparse tensor compiler and comparing them with those generated with the default TACO schedules. Our results show that our approach reduces the scheduling space by orders of magnitude and that the generated kernels perform asymptotically better than those generated using the default schedules.

show abstract

“…To support such computations, our technique emits code that uses a set of loops to iterate over intersections or unions of the operands' nonzeros and compute with those nonzeros, as Figure 2c demonstrates for instance. Kjolstad et al [22] and Henry et al [17] describe how such loops can be generated assuming it is possible to enumerate the stored nonzeros of each operand. However, while Chou et al [11] show how code to perform such enumeration can be emitted for operands that are stored in static, array-based formats, their technique does not support dynamic, pointer-based sparse tensor formats.…”

Section: Generating Iteratorsmentioning

confidence: 99%

Dynamic Sparse Tensor Algebra Compilation

Chou¹,

Amarasinghe²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper shows how to generate efficient tensor algebra code that compute on dynamic sparse tensors, which have sparsity structures that evolve over time. We propose a language for precisely specifying recursive, pointer-based data structures, and we show how this language can express a wide range of dynamic data structures that support efficient modification, such as linked lists, binary search trees, and B-trees. We then describe how, given high-level specifications of such data structures, a compiler can generate code to efficiently iterate over and compute with dynamic sparse tensors that are stored in the aforementioned data structures. Furthermore, we define an abstract interface that captures how nonzeros can be inserted into dynamic data structures, and we show how this abstraction guides a compiler to emit efficient code that store the results of sparse tensor algebra computations in dynamic data structures.We evaluate our technique and find that it generates efficient dynamic sparse tensor algebra kernels. Code that our technique emits to compute the main kernel of the PageRank algorithm is 1.05× as fast as Aspen, a state-of-the-art dynamic graph processing framework. Furthermore, our technique outperforms PAM, a parallel ordered (key-value) maps library, by 7.40× when used to implement element-wise addition of a dynamic sparse matrix to a static sparse matrix.

show abstract

Compilation of sparse array programming models

Cited by 21 publications

References 21 publications

Compiler Support for Sparse Tensor Computations in MLIR

Compiler Support for Sparse Tensor Computations in MLIR

An Asymptotic Cost Model for Autoscheduling Sparse Tensor Programs

Dynamic Sparse Tensor Algebra Compilation

Contact Info

Product

Resources

About