Set and Relation Manipulation for the Sparse Polyhedral Framework

Strout, Michelle Mills; Georg, Geri; Olschanowsky, Catherine

doi:10.1007/978-3-642-37658-0_5

Cited by 16 publications

(10 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Implementation of the automatic wavefront parallelization of sparse codes uses a polyhedral transformation and code generation framework comprised of CHiLL [31], IEGenLib [32], Omega+ [28] and Codegen+ [27]. The generated code is integrated with the SpMP [33] open source library for parallelizing sparse computations.…”

Section: Methodsmentioning

confidence: 99%

Automating Wavefront Parallelization for Sparse Matrix Computations

Venkat¹,

Mohammadi²,

Park³

et al. 2016

SC16: International Conference for High Performance Computing, Networking, Storage and Analysis

Self Cite

View full text Add to dashboard Cite

This paper presents a compiler and runtime framework for parallelizing sparse matrix computations that have loopcarried dependences. Our approach automatically generates a runtime inspector to collect data dependence information and achieves wavefront parallelization of the computation, where iterations within a wavefront execute in parallel, and synchronization is required across wavefronts. A key contribution of this paper involves dependence simplification, which reduces the time and space overhead of the inspector. This is implemented within a polyhedral compiler framework, extended for sparse matrix codes. Results demonstrate the feasibility of using automaticallygenerated inspectors and executors to optimize ILU factorization and symmetric Gauss-Seidel relaxations, which are part of the Preconditioned Conjugate Gradient (PCG) computation. Our implementation achieves a median speedup of 2.97× on 12 cores over the reference sequential PCG implementation, significantly outperforms PCG parallelized using Intel's Math Kernel Library (MKL), and is within 6% of the median performance of manually-parallelized PCG.

show abstract

Section: Methodsmentioning

confidence: 99%

Automating Wavefront Parallelization for Sparse Matrix Computations

Venkat¹,

Mohammadi²,

Park³

et al. 2016

SC16: International Conference for High Performance Computing, Networking, Storage and Analysis

Self Cite

View full text Add to dashboard Cite

show abstract

“…Sparse code, however, involves nested indirect array accesses. Recent work has to extend the polyhedral model to these situations [Belaoucha et al 2010;Strout et al 2012;Venkat et al 2015Venkat et al , 2016, using a combination of compile-time and runtime techniques, but the space of loop nests on nested indirect array accesses is complicated, and it difficult for compilers to determine when linear-algebraic optimizations are applicable to the operations that the code represents. Our workspace optimization applies to sparse tensor algebra at the concrete index notation level, before sparse code is generated, which makes it possible to perform aggressive optimizations and convenient to reason about legality.…”

Section: Related Workmentioning

confidence: 99%

Tensor Algebra Compilation with Workspaces

Kjølstad¹,

Ahrens²,

Kamil

et al. 2019

2019 IEEE/ACM International Symposium on Code Generation and Optimization (CGO)

View full text Add to dashboard Cite

This paper shows how to optimize sparse tensor algebraic expressions by introducing temporary tensors, called workspaces, into the resulting loop nests. We develop a new intermediate language for tensor operations called concrete index notation that extends tensor index notation. Concrete index notation expresses when and where sub-computations occur and what tensor they are stored into. We then describe the workspace optimization in this language, and how to compile it to sparse code by building on prior work in the literature.We demonstrate the importance of the optimization on several important sparse tensor kernels, including sparse matrix-matrix multiplication (SpMM), sparse tensor addition (SpAdd), and the matricized tensor times Khatri-Rao product (MTTKRP) used to factorize tensors. Our results show improvements over prior work on tensor algebra compilation and brings the performance of these kernels on par with state-of-the-art hand-optimized implementations. For example, SpMM was not supported by prior tensor algebra compilers, the performance of MTTKRP on the nell-2 data set improves by 35%, and MTTKRP can for the first time have sparse results.Additional

show abstract

“…Section 5 clearly demonstrates that the framework developed here is more scalable than such trace-based approaches. The Sparse Polyhedral Framework [23,43] also provides a uni-fied framework to express affine and irregular parts of the code by representing indirection array access using uninterpreted function symbols (UFS). Still, transformations and code generation process within this framework requires asserting properties of these UFS (e.g., invertibility) at compile time, which is usually not possible.…”

Section: Related Workmentioning

confidence: 99%

“…For example, state-of-the-art polyhedral compilers (e.g., PolyOpt [29]) can optimize the computation within loop i in Listing 1, while the corresponding loop in Listing 2 is outside the scope of such compilers. The Sparse Polyhedral Framework [23,43] aims to handle programs with a mix of affine and non-affine code regions by extending the polyhedral compilation framework. It uses uninterpreted function symbols (UFS) to model indirect accesses.…”

Section: Introductionmentioning

confidence: 99%

Distributed memory code generation for mixed Irregular/Regular computations

et al. 2015

View full text Add to dashboard Cite

Many applications feature a mix of irregular and regular computational structures. For example, codes using adaptive mesh refinement (AMR) typically use a collection of regular blocks, where the number of blocks and the relationship between blocks is irregular. The computational structure in such applications generally involves regular (affine) loop computations within some number of innermost loops, while outer loops exhibit irregularity due to datadependent control flow and indirect array access patterns. Prior approaches to distributed memory parallelization do not handle such computations effectively. They either target loop nests that are completely affine using polyhedral frameworks, or treat all loops as irregular. Consequently, the generated distributed memory code contains artifacts that disrupt the regular nature of previously affine innermost loops of the computation. This hampers subsequent optimizations to improve on-node performance.We propose a code generation framework that can effectively transform such applications for execution on distributed memory systems. Our approach generates distributed memory code which preserves program properties that enable subsequent polyhederal optimizations. Simultaneously, it addresses a major memory bottleneck of prior techniques that limits the scalability of the generated code. The effectiveness of the proposed framework is demonstrated on computations that are mixed regular/irregular, completely regular, and completely irregular.

show abstract

Set and Relation Manipulation for the Sparse Polyhedral Framework

Cited by 16 publications

References 34 publications

Automating Wavefront Parallelization for Sparse Matrix Computations

Automating Wavefront Parallelization for Sparse Matrix Computations

Tensor Algebra Compilation with Workspaces

Distributed memory code generation for mixed Irregular/Regular computations

Contact Info

Product

Resources

About