Generating efficient tiled code for distributed memory machines

Tang, Peiyi; Xue, Jingling

doi:10.1016/s0167-8191(00)00040-5

Cited by 11 publications

(3 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Tang and Xue [19] presented a method for tiling SOR by applying skewing and tiling for distributed memory machines. Goumas et al [4] later continued this line of investigation by focusing on the parallelization of 2D iteration spaces that result from the discretization of PDEs.…”

Section: A Parallel Mlssor Methods For Gpgpusmentioning

confidence: 99%

Toward Harnessing DOACROSS Parallelism for Multi-GPGPUs

Peng

Wan

Zhang

et al. 2010

2010 39th International Conference on Parallel Processing

Self Cite

View full text Add to dashboard Cite

Abstract-To exploit the full potential of GPGPUs for generalpurpose computing, DOACR parallelism abundant in scientific and engineering applications must be harnessed. However, the presence of cross-iteration data dependences in DOACR loops poses an obstacle to execute their computations concurrently using a massive number of fine-grained threads. This work focuses on iterative PDE solvers rich in DOACR parallelism to identify optimization principles and strategies that allow their efficient mapping to GPGPUs. Our main finding is that certain DOACR loops can be accelerated further on GPGPUs if they are algorithmically restructured (by a domain expert) to be more amendable to GPGPU parallelization, judiciously optimized (by the compiler), and carefully tuned by a performance-tuning tool.We substantiate this finding with a case study by presenting a new parallel SSOR method that admits more efficient data-parallel SIMD execution than red-black SOR on GPGPUs. Our solution is obtained non-conventionally, by starting from a K-layer SSOR method and then parallelizing it by applying a non-dependencepreserving scheme consisting of a new domain decomposition technique followed by a generalized loop tiling. Despite its relatively slower convergence, our new method outperforms red-black SOR by making a better balance between data reuse and parallelism and by trading off convergence rate for SIMD parallelism. Our experimental results highlight the importance of synergy between domain experts, compiler optimizations and performance tuning in maximizing the performance of applications, particularly PDE-based DOACR loops, on GPGPUs.

show abstract

Section: A Parallel Mlssor Methods For Gpgpusmentioning

confidence: 99%

Toward Harnessing DOACROSS Parallelism for Multi-GPGPUs

Peng

Wan

Zhang

et al. 2010

2010 39th International Conference on Parallel Processing

Self Cite

View full text Add to dashboard Cite

show abstract

“…Tang and Xue in [22] addressed the same issues for rectangularly tiled iteration spaces. We can also generate efficient data parallel code for non-rectangular tiles without imposing any further complexity.…”

Section: Data Parallel Code Generationmentioning

confidence: 98%

Automatic parallel code generation for tiled nested loops

Goumas¹,

Drosinos²,

Athanasaki³

et al. 2004

Proceedings of the 2004 ACM Symposium on Applied Computing

View full text Add to dashboard Cite

This paper presents an overview of our work, concerning a complete end-to-end framework for automatically generating message passing parallel code for tiled nested for-loops. It considers general parallelepiped tiling transformations and general convex iteration spaces. We address all problems regarding both the generation of sequential tiled code and its parallelization. We have implemented our techniques in a tool which automatically generates MPI parallel code and conducted several series of experiments, concerning the compilation time of our tool, the efficiency of the generated code and the speedup attained on a cluster of PCs. Apart from confirming the value of our techniques, our experimental results show the merit of general parallelepiped tiling transformations and verify previous theoretical work on scheduling-optimal tile shapes.

show abstract

“…Loop tiling is beneficial for parallel computers with hierarchical memory: computers with both shared and distributed memory (Xue, 2012;Tang and Xue, 2000) as…”

Section: Introductionmentioning

confidence: 99%

Tiling arbitrarily nested loops by means of the transitive

Bielecki

Palkowski

2016

International Journal of Applied Mathematics and Computer Science

View full text Add to dashboard Cite

A novel approach to generation of tiled code for arbitrarily nested loops is presented. It is derived via a combination of the polyhedral and iteration space slicing frameworks. Instead of program transformations represented by a set of affine functions, one for each statement, it uses the transitive closure of a loop nest dependence graph to carry out corrections of original rectangular tiles so that all dependences of the original loop nest are preserved under the lexicographic order of target tiles. Parallel tiled code can be generated on the basis of valid serial tiled code by means of applying affine transformations or transitive closure using on input an inter-tile dependence graph whose vertices are represented by target tiles while edges connect dependent target tiles. We demonstrate how a relation describing such a graph can be formed. The main merit of the presented approach in comparison with the well-known ones is that it does not require full permutability of loops to generate both serial and parallel tiled codes; this increases the scope of loop nests to be tiled.

show abstract

Generating efficient tiled code for distributed memory machines

Cited by 11 publications

References 38 publications

Toward Harnessing DOACROSS Parallelism for Multi-GPGPUs

Toward Harnessing DOACROSS Parallelism for Multi-GPGPUs

Automatic parallel code generation for tiled nested loops

Tiling arbitrarily nested loops by means of the transitive

Contact Info

Product

Resources

About