Automatic Data Partitioning on Distributed Memory Multiprocessors

Gupta, Manish; Banerjee, P.

doi:10.1109/dmcc.1991.633082

Cited by 30 publications

(27 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This problem is related to the problem of choosing good data decompositions in languages such as High Performance Fortran [34]. We expect that the techniques for layout optimization developed in the vector and parallel computing context (e.g., [11,25,31,32,41]) can be adapted to the hierarchical memory situation. Table 2 describes benchmark suite that we use to evaluate nonlinear array layout functions.…”

Section: Discussionmentioning

confidence: 99%

“…This would further reduce the number of conversions between canonical and non-standard layouts. This is essentially the approach to data parallelism taken in High Performance Fortran [34], and techniques for automatic data mapping developed in that context (e.g., [11,25,31,32]) can equally well be applied to a uniprocessor environment to improve memory hierarchy performance.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Nonlinear array layouts for hierarchical memory systems

Chatterjee

Jain

Lebeck

et al. 1999

Proceedings of the 13th International Conference on Supercomputing

126

142

View full text Add to dashboard Cite

Programming languages that provide multidimensional arrays and a flat linear model of memory must implement a mapping between these two domains to order array elements in memory. This layout function is fixed at language definition time and constitutes an invisible, non-programmable array attribute. In reality, modern memory systems are architecturally hierarchical rather than flat, with substantial differences in performance among different levels of the hierarchy. This mismatch between the model and the true architecture of memory systems can result in low locality of reference and poor performance. Some of this loss in performance can be recovered by re-ordering computations using transformations such as loop tiling. We explore nonlinear array layout functions as an additional means of improving locality of reference. For a benchmark suite composed of dense matrix kernels, we show by timing and simulation that two specific layouts (4D and Morton) have low implementation costs (2-5% of total running time) and high performance benefits (reducing execution time by factors of 1.1-2.5); that they have smooth performance curves, both across a wide range of problem sizes and over representative cache architectures; and that recursion-based control structures may be needed to fully exploit their potential.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Nonlinear array layouts for hierarchical memory systems

Chatterjee

Jain

Lebeck

et al. 1999

Proceedings of the 13th International Conference on Supercomputing

126

142

View full text Add to dashboard Cite

show abstract

“…The polyhedral method is another branch of compiler optimization that seeks to speed up parallel programs on distributed memory architectures [10,6,8,11,12,20]. Its strength is that it can find sequences of transformations in one step, without searching the entire space of transformations.…”

Section: Related Workmentioning

confidence: 99%

Affine Loop Optimization Based on Modulo Unrolling in Chapel

Sharma

Smith

Koehler

et al. 2014

Proceedings of the 8th International Conference on Partitioned Global Address Space Programming Models

View full text Add to dashboard Cite

This paper presents modulo unrolling without unrolling (modulo unrolling WU), a method for message aggregation for parallel loops in message passing programs that use affine array accesses in Chapel, a Partitioned Global Address Space (PGAS) parallel programming language. Messages incur a non-trivial run time overhead, a significant component of which is independent of the size of the message. Therefore, aggregating messages improves performance. Our optimization for message aggregation is based on a technique known as modulo unrolling, pioneered by Barua [3], whose purpose was to ensure a statically predictable single tile number for each memory reference for tiled architectures, such as the MIT Raw Machine [18]. Modulo unrolling WU applies to data that is distributed in a cyclic or block-cyclic manner. In this paper, we adapt the aforementioned modulo unrolling technique to the difficult problem of efficiently compiling PGAS languages to message passing architectures. When applied to loops and data distributed cyclically or blockcyclically, modulo unrolling WU can decide when to aggregate messages thereby reducing the overall message count and runtime for a particular loop. Compared to other methods, modulo unrolling WU greatly simplifies the complex problem of automatic code generation of message passing code. It also results in substantial performance improvement compared to the non-optimized Chapel compiler.To implement this optimization in Chapel, we modify the leader and follower iterators in the Cyclic and Block Cyclic data distribution modules. Results were collected that compare the performance of Chapel programs optimized with modulo unrolling WU and Chapel programs using the existing Chapel data distributions. Data collected on a tenlocale cluster show that on average, modulo unrolling WU used with Chapel's Cyclic distribution results in 64 percent fewer messages and a 36 percent decrease in runtime for our suite of benchmarks. Similarly, modulo unrolling WU used with Chapel's Block Cyclic distribution results in 72 percent fewer messages and a 53 percent decrease in runtime.

show abstract

“…Thus, Wholey considers a restricted version of the dynamic problem.) Gupta [8] uses heuristic methods to determine the distribution parameters. He analyzes communication patterns to determine whether block or cyclic distributions are preferable.…”

Section: Previous Workmentioning

confidence: 99%