On supernode transformation with minimized total running time

Abstract

With the objective of minimizing the total execution time of a parallel program on a distributed memory parallel computer, this paper discusses how to nd an optimal supernode size and optimal supernode relative side lengths of a supernode transformation (also known as tiling). We identify three parameters of supernode transformation: supernode size, relative side lengths, and cutting hyperplane directions. For algorithms with perfectly nested loops and uniform dependencies, for su ciently large supernodes and number of processors, and for the case where multiple supernodes are mapped to a single processor, we give an order n polynomial whose real positive roots include the optimal supernode size. For two special cases: (1) two dimensional algorithm problems and (2) n-dimensional algorithm problems where the communication cost is dominated by the startup penalty and therefore, can be approximated by a constant, we give a closed form expression for the optimal supernode size, which is independent of the supernode relative side lengths and cutting hyperplanes. For the case where the algorithm iteration index space and the supernodes are hyperrectangular, we give closed form expressions for the optimal supernode relative side lengths. Our experiment shows a good match of the closed form expressions with experimental data.

Hyperplane Grouping and Pipelined Schedules: How to Execute Tiled Loops Fast on Clusters of SMPs

“…In [24], Hodzic and Shang have presented a scheme for scheduling loops that have been transformed through a supernode transformation. Their approach is to minimize total execution time, as follows: Firstly, the optimal tiling matrix H is determined and then it is applied to the original Iteration Space.…”

“…, 1). Notice, also, that this scheduling vector is not the same with Hodzic's [24] scheduling vector, since we are now referring to groups, while Hodzic was scheduling tiles.…”

“…For each schedule, we are interested in the overall minimum execution time achieved at an optimally selected tile height (see [22,24,43]). The experimental results, shown in Figures 18-19, illustrate that, in every case, non-blocking communication is preferable to blocking communication and hyperplane grouping is preferable to vertical grouping.…”

“…This problem will not concern our scheduling, but it will mean that the communication architecture is too slow to exploit all the computation power of the computing system. Then, it would be better not to use all the nodes available, as implied in [24]. If we add more CPUs inside each SMP node, we may again cut the initial iteration space into smaller tiles.…”

“…Hodzic and Shang [24] proposed a method to correlate optimal tile size and shape, based on overall completion time reduction. Their approach considers a straightforward time schedule, where each processor executes all tiles along a specific dimension, by interleaving computation and communication phases.…”