Advanced compilation techniques in the PARADIGM compiler for distributed-memory multicomputers

Abstract

The PARADIGM compiler project provides an automated means to parallelize programs, written in a serial programming model, for efficient execution on distributed-memory muiticomputers. A previous implementation of the compiler based on the PTD representation allowed symbolic array sizes, affine loop bounds and array subscripts, and variable number of processors, provided that arrays were singleor multi-dimensionally block distributed. The techniques .presented here extend the compiler to also accept multidimensional cyclic and block-cyclic distributions within a uniform symbolic framework. These extensions demand more sophisticated symbolic manipulation capabilities. A novel aspect of our approach is to meet this demand by interfacing PARADIGM with a powerful off-the-shelf symbolic package. Mathematical™. This paper describes some of the Mathematical™ routines that performs various transformations, shows how they are invoked and used by the compiler . to overcome the new challenges, and presents experimental results for code involving cyclic and block-cyclic arrays as evidence of the feasibility of the approach.

Experiences with Enumeration of Integer Projections of Parametric Polytopes

“…These counts are used to drive optimizations such as increasing parallelism [38], minimizing memory size [1,2,27,38,42], estimating worst case execution time [28], increasing cache effectiveness [6,16], high-level transformations for DSP applications [23], converting software loops into parallel hardware implementations [5,18,22,25,38] and minimizing communication overhead in distributed applications [9,26,37]. In many of these optimizations, the objects or events to be counted are modeled as the integer solutions to systems of linear inequalities, i.e., as the elements of a set S = { x ∈ Z d | Ax + c ≥ 0 }, with A ∈ Z n×d and c ∈ Z n .…”

“…Computing the number of elements in such a set is, amongst others, needed in the program analyses described in [1,2,5,6,9,12,16,25,26,37,42]. Practical examples are discussed in an extended version of this paper, see [39].…”

Compiler Construction

“…Examples include counting the number of calculations, accessed memory locations or statement executions in a loop nest or parts thereof [6,21,23,28,29,38]; calculating the number of cache misses in a loop [12,16,24]; computing the number of dynamically allocated bytes [11]; enumerating the number of live array elements at a given iteration (i, j) [27,42]; counting how many parallel processing elements can be used when executing a loop on an FPGA [5,22,25] and computing the amount of communication for a given schedule of parallel tasks on a distributed computing system [9,26,37].…”

PARADIGM (version 2.0): a new HPF compilation system

“…Though PTDs are attractive from the standpoint of supporting symbolic array sizes, variable number of processors and multidimensional distributions, their utility is restricted to BLOCK distributions alone. In fact, the original version of PARADIGM provided support for only BLOCK distributions and then subsequent work resulted in limited assistance for the CYCLICB case through the Fourier-Motzkin Elimination (FME) technique [12]. The challenge arises in attempting to devise a uniform scheme that provides simultaneous support for the general CYCLICB distribution.…”

“…Some critics of our approach to using Mathematica to perform symbolic analysis [12] believe that having an expensive communication conduit in the form of MathLink is an infeasible approach for realizing a fast compiler architecture. Agreeably in the past, the MathLink channel has been the hindering factor with regard to compilation times, since this constitutes the only way an external program can interact with the Mathematica kernel.…”