Fast Runtime Block Cyclic Data Redistribution on Multiprocessors

Prylli, Loïc; Tourancheau, Bernard

doi:10.1006/jpdc.1997.1351

Cited by 45 publications

(37 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In many applications, data redistribution may be needed if elements of a data set are inserted or deleted at the end of the array. In particular, algorithms to redistribute data using a new block size exist for the Block Cyclic distribution [14,19]. If an application uses a dynamic data set with elements that are appended, a Cyclic or Block Cyclic distribution is superior to Block because new elements are added to the locale that follows the cyclic or block-cyclic pattern.…”

Section: Chapel's Data Distributionsmentioning

confidence: 99%

Affine Loop Optimization Based on Modulo Unrolling in Chapel

Sharma

Smith

Koehler

et al. 2014

Proceedings of the 8th International Conference on Partitioned Global Address Space Programming Models

View full text Add to dashboard Cite

This paper presents modulo unrolling without unrolling (modulo unrolling WU), a method for message aggregation for parallel loops in message passing programs that use affine array accesses in Chapel, a Partitioned Global Address Space (PGAS) parallel programming language. Messages incur a non-trivial run time overhead, a significant component of which is independent of the size of the message. Therefore, aggregating messages improves performance. Our optimization for message aggregation is based on a technique known as modulo unrolling, pioneered by Barua [3], whose purpose was to ensure a statically predictable single tile number for each memory reference for tiled architectures, such as the MIT Raw Machine [18]. Modulo unrolling WU applies to data that is distributed in a cyclic or block-cyclic manner. In this paper, we adapt the aforementioned modulo unrolling technique to the difficult problem of efficiently compiling PGAS languages to message passing architectures. When applied to loops and data distributed cyclically or blockcyclically, modulo unrolling WU can decide when to aggregate messages thereby reducing the overall message count and runtime for a particular loop. Compared to other methods, modulo unrolling WU greatly simplifies the complex problem of automatic code generation of message passing code. It also results in substantial performance improvement compared to the non-optimized Chapel compiler.To implement this optimization in Chapel, we modify the leader and follower iterators in the Cyclic and Block Cyclic data distribution modules. Results were collected that compare the performance of Chapel programs optimized with modulo unrolling WU and Chapel programs using the existing Chapel data distributions. Data collected on a tenlocale cluster show that on average, modulo unrolling WU used with Chapel's Cyclic distribution results in 64 percent fewer messages and a 36 percent decrease in runtime for our suite of benchmarks. Similarly, modulo unrolling WU used with Chapel's Block Cyclic distribution results in 72 percent fewer messages and a 53 percent decrease in runtime.

show abstract

Section: Chapel's Data Distributionsmentioning

confidence: 99%

Affine Loop Optimization Based on Modulo Unrolling in Chapel

Sharma

Smith

Koehler

et al. 2014

Proceedings of the 8th International Conference on Partitioned Global Address Space Programming Models

View full text Add to dashboard Cite

show abstract

“…the computation-to-communication ratio. Because this granularity changes from one computational kernel to the other, moving from a CYCLIC(r) distribution over p processors to a CYCLIC(s) distribution over q processors is a very useful redistribution procedure, which has been implemented using a caterpillar algorithm in ScaLAPACK [34]. Several papers, including [23,39,14,33,19,11,24], have dealt with various optimizations of this redistribution procedure.…”

Section: Related Workmentioning

confidence: 99%

Data Redistribution Algorithms for Heterogeneous Processor Rings

Renard¹,

Robert

Vivien

2006

The International Journal of High Performance Computing Applica

View full text Add to dashboard Cite

We consider the problem of redistributing data on homogeneous and heterogeneous ring of processors. The problem arises in several applications, each time after that a load-balancing mechanism is invoked (but we do not discuss the load-balancing mechanism itself). We provide algorithms that aim at optimizing the data redistribution, both for uni-directional and bi-directional rings, and we give complete proofs of correctness. One major contribution of the paper is that we are able to prove the optimality of the proposed algorithms in all cases except that of a bi-directional heterogeneous ring, for which the problem remains open. Keywords: Heterogeneous rings, data redistribution algorithms, load-balancing RésuméDans ce rapport, nous nous intéressons au problème de redistribution de données sur des anneaux de processeurs homogènes et hétérogènes. Ce problème surgit dans plusieurs applications, après chaque phase d'équilibrage de charge (nous ne discutons pas ici du mécanisme d'équilibrage de charge lui-même). Nous proposons des algorithmes qui visentà optimiser la redistribution de données pour des anneaux unidirectionnels et bidirectionnels, et nous donnons toutes les preuves de correction de ces algorithmes. Une des contributions principales de ce rapport est que nous pouvons prouver l'optimialité des algorithmes proposés dans tous les cas, sauf dans le cas d'un anneau hétérogène bidirectionnel, pour lequel le problème reste ouvert.

show abstract

“…Based on the intersections, the send/receive processor/data sets can be determined and general redistribution algorithms can be devised. Prylli and Touranchean [21] proposed a runtime scan algorithm for BLOCK-CYCLIC array redistribution. Their approach has the same time complexity as that proposed in [23] but has a simple basic operation compared to that proposed in [23].…”

Section: Related Workmentioning

confidence: 99%

“…In some algorithms, such as multidimensional fast Fourier transform [29], the Alternative Direction Implicit (ADI) method for solving two-dimensional diffusion equations, and linear algebra solvers [21], an array distribution that is well suited for one phase may not be good for a subsequent phase in terms of performance. Array redistribution is required for those algorithms at runtime.…”

Section: Introductionmentioning

confidence: 99%

A generalized processor mapping technique for array redistribution

Hsu

Chung²,

Yang³

et al. 2001

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

AbstractÐIn many scientific applications, array redistribution is usually required to enhance data locality and reduce remote memory access in many parallel programs on distributed memory multicomputers. Since the redistribution is performed at runtime, there is a performance trade-off between the efficiency of the new data decomposition for a subsequent phase of an algorithm and the cost of redistributing data among processors. In this paper, we present a generalized processor mapping technique to minimize the amount of data exchange for BLOCK-CYCLIC(kr) to BLOCK-CYCLIC(r) array redistribution and vice versa. The main idea of the generalized processor mapping technique is first to develop mapping functions for computing a new rank of each destination processor. Based on the mapping functions, a new logical sequence of destination processors can be derived. The new logical processor sequence is then used to minimize the amount of data exchange in a redistribution. The generalized processor mapping technique can handle array redistribution with arbitrary source and destination processor sets and can be applied to multidimensional array redistribution. We present a theoretical model to analyze the performance improvement of the generalized processor mapping technique. To evaluate the performance of the proposed technique, we have implemented the generalized processor mapping technique on an IBM SP2 parallel machine. The experimental results show that the generalized processor mapping technique can provide performance improvement over a wide range of redistribution problems.

show abstract

Fast Runtime Block Cyclic Data Redistribution on Multiprocessors

Cited by 45 publications

References 15 publications

Affine Loop Optimization Based on Modulo Unrolling in Chapel

Affine Loop Optimization Based on Modulo Unrolling in Chapel

Data Redistribution Algorithms for Heterogeneous Processor Rings

A generalized processor mapping technique for array redistribution

Contact Info

Product

Resources

About