Hyperplane Grouping and Pipelined Schedules: How to Execute Tiled Loops Fast on Clusters of SMPs

Abstract

Abstract. This paper proposes a novel approach for the parallel execution of tiled Iteration Spaces onto a cluster of SMP PC nodes. Each SMP node has multiple CPUs and a single memory mapped PCI-SCI Network Interface Card. We apply a hyperplane-based grouping transformation to the tiled space, so as to group together independent neighboring tiles and assign them to the same SMP node. In this way, intranode (intragroup) communication is annihilated. Groups are atomically executed inside each node. Nodes exchange data between successive group computations. We schedule groups much more efficiently by exploiting the inherent overlapping between communication and computation phases among successive atomic group executions. The applied non-blocking schedule resembles a pipelined datapath, where group computation phases are overlapped with communication ones, instead of being interleaved with them. Our experimental results illustrate that the proposed method outperforms previous approaches involving blocking communication or conventional grouping schemes.

Parallel loop generation and scheduling

“…To compute the last tile coordinates, relation (2) can be applied to the last iteration point, UB = (ub 1 , ub 2 , . .…”

“…11 An algorithm for computing the completion time of a scheduling total execution time or the completion time of a scheduling is given. In this algorithm it is assumed that the computation time of each tile and communication time between dependent tiles are overlapped [2,8,14]. Also, message passing between processors is performed in parallel and processors start to execute all the tiles on the same wavefront at the same time.…”

Automatic code generation for distributed memory architectures in the polytope model

“…Athanasaki et al describe another approach that also uses tiling to reduce communication for distributed memory based clusters [2]. In their approach, an additional tiling transformation is used for aggregating processor tiles along certain hyperplanes into so-called groups, which can be executed efficiently by exploiting the availability of communicationfree shared memory processors on each node in the cluster.…”