On time optimal supernode shape

Abstract

With the objective of minimizing the total execution time of a parallel program on a distributed memory parallel computer, this paper discusses the selection of an optimal supernode shape of a supernode transformation (also known as tiling). We identify three parameters of a supernode transformation: supernode size, relative side lengths, and cutting hyperplane directions. For supernode transformations on algorithms with perfectly nested loops and uniform dependencies, we prove the optimality of a constant linear schedule vector and give a necessary and sufficient condition for optimal relative side lengths. We also prove that the total running time is minimized by a cutting hyperplane direction matrix from a particular subset of all valid directions and we discuss the cases where this subset is unique. The results are derived in continuous space and should be considered approximate. Our model does not include cache effects and assumes an unbounded number of available processors, the communication cost approximated by a constant, uniform dependences, and loop bounds known at compile time. A comprehensive example is discussed with an application of the results to the Jacobi algorithm.

Automatic parallel code generation for tiled nested loops

“…To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SAC '04, March [14][15][16][17]2004, Nicosia, Cyprus. densome task, since one has to take into consideration all special characteristics of the underlying hardware (processor speed, cache and cache-line sizes, interconnection network bandwidth and latency etc.).…”

An optimal scheduling scheme for tiling in distributed systems

“…Högstedt et al in [11] is using the critical execution path to propose a model for calculating the execution time of tiling and define the optimal tile shape. Hodzic et al in [10] propose a new framework of defining tiling parameters for an arbitrary iteration space with dependences based upon minimizing theoretical execution time. Goumas et al in [9] propose efficient code generation algorithms for tiled iteration spaces in message passing distributed systems.…”

“…IMPLEMENTATION AS A TILING SCHEME In this section we define a new kind of tiling scheme in two dimensional iteration spaces with variable edge lengths. On one dimension the tile size follows the trapezoid arithmetic progression defined in (8) and on the other dimension it follows the geometric progression defined in (10). We term this type of tiling as Trapezoid-Geometric Scheduling (TGS).…”

“…As an example consider the case where P = 4, F = 128 and L = 11. Using the values of F, L, N 1 and N 2 we compute λ and n 2 (1) according to formulas in (10) and (12 Figure 6 displays an experimental comparison of TGS against CS and TS in terms of execution performance and speedup. The graphs in Figure 6 display the execution times and the produced speedup.…”

Hyperplane Grouping and Pipelined Schedules: How to Execute Tiled Loops Fast on Clusters of SMPs

“…Later, Ramanujam and Sadayappan in [38] showed the equivalence between the problem of finding a set of extreme vectors for a given set of dependence vectors and the problem of finding a tiling transformation H that produce valid, deadlock-free tiles. The problem of determining the optimal shape was surveyed, and more accurate conditions were also given by others, as in [12,16,[25][26][27][28]47].…”