Compilation of a specialized functional language for massively parallel computers

Abstract

We propose a parallel specialized language that ensures portable and cost-predictable implementations on parallel computers. The language is basically a first-order, recursion-less, strict functional language equipped with a collection of higher-order functions or skeletons. These skeletons apply on (nested) vectors and can be grouped into four classes: computation, reorganization, communication and mask skeletons. The compilation process is described as a series of transformations and analyses leading to SPMD-like functional programs which can be directly translated into real parallel code. The language restrictions enforce a programming discipline whose benefit is to allow a static, symbolic and accurate cost analysis. The parallel cost takes into account both load balancing and communications, and can be statically evaluated even when the actual size of vectors or the number of processors are unknown. It is used to automatically select the best data distribution among a set of standard distributions. Interestingly, this work can be seen as a cross-fertilization between techniques developed within the FORTRAN parallelization, skeleton and functional programming communities.

Skeleton-based parallel programming: Functional and parallel semantics in a single shot

“…It also allows to reason about program equivalence, in terms of the results computed, or to define semantics-preserving program transformations [18][19][20]. These transformations can also be driven by some kind of analytical performance models associated with skeletons, in such a way that only those rewritings leading to efficient implementations of the skeleton code are considered [18,[21][22][23][24]. For instance, one can easily derive that the following two programs actually compute the same result:…”