A monadic calculus for parallel costing of a functional language of arrays

Abstract

VEC is a higher-order functional language of nested arrays, which includes a general folding operation. Static computation of the shape, of its programs is used to support a compositional cost calculus based:on a cost monad. This, in turn, is based on a cost algebra, whose operations may be customized to handle different cost regimes, especially for parallel programming. We present examples based on sequential costing and~on the PRAM model of parallel computation. The latter has been implemented in Haskell, and applied to some linear algebra examples. I n t r o d u c t i o nSecond-order combinators such as map, fold and zip provide programmers with a concise, abstract language for writing skeletons for implicitly parallel programs, as in [Ski94], but there is a hitch. These combinators are defined for list programs (see [BW88]), but efficient implementations (which is the point of parallelism, after all) are based on arrays. This disparity becomes acute when working with nested arrays, which are rarely supported in parallel practice. NESL [BCH+94] is a notable exception, but still does not support folding over them. Our approach is to redevelop the theory of arrays to support the combinators while retaining constant-time access. We illustrate the efficacy of this approach by implementing static estimates of shapes and parallel work, as compared to, say, the dynamically obtained estimates for NESL [BG96].Efficient array access is based on direct access to the storage of array entries, typically in contiguous blocks of memory. List storage, however, is by allocating cons cells during evaluation, which introduces significant access costs. Our approach to arrays uses the syntax of lists, so that combinators can be introduced in the functional style, but constrained so that a compiler can still determine the length, or more generally the shape, of any array expression. This process of shape analysis can be thought of as compile-time/run-time separation in a two-level operational semantics INN92]. Success requires that the program be shapely, that is, the shape of the result is determined by those of its inputs, as is typical in, say, linear algebra. Shapeliness, and successful shape analysis, require a new approach, which we have illustrated in a small functional language, VEC.1. This operation is shapely: the result is an integer, a datum whose shape is trivial and therefore unaffected by the condition.

Static Performance Prediction of Skeletal Parallel Programs

“…Our analysis predicts the behaviour of these programs when compiled for an SPMD model, choosing BSP [7,15,17] to introduce parallelism because of its explicit support for dependably stable performance analysis across diverse architectures. Our work extends previously reported results in the field [6,11].…”

“…Static shape analysis to support compilation and cost prediction for parallel programs was originally suggested by Jay [8] and first applied in detail to the cost analysis of Vec, a small shapely functional language [11]. This work used the tightly synchronised, uniform access cost, shared memory PRAM model as its target architecture.…”

“…Our language is based on the small shapely functional language Vec defined in [11], with the addition of new parallel skeletons. We summarise its features here.…”

“…The cost calculus in [11] computes PRAM cost and so does not model communication effects. In [6] we extended the analysis to detect inter-processor communication costs in terms of the BSP cost model.…”

“…Our cost calculus has been implemented as a Haskell program based on that developed for the PRAM by Jay's group [11]. It consists of nearly one thousand lines of Haskell and takes a few seconds to analyse the short examples presented here.…”