Compiling nested data-parallel programs for shared-memory multiprocessors

Abstract

While data parallelism is well suited from algorithmic, architectural, and linguistic considerations to serve as a basis for portable parallel programming, its characteristic fine-grained parallelism makes the efficient implementation of data-parallel languages on MIMD machmes a challenging task. The design, implementation, and evaluation of an optimizmg compiler are presented for an applicative nested data-parallel language called VCODE targeted at the Encore Multimax, a shared-memory multiprocessor The source language supports nested aggregate data types; aggregate operations including elementwiseformsj scans, reductions, andpermutations; and conditionals and recursion for control flow. A small set of graph-theoretic compile-time optimizations reduce the overheads on MIMD machines in several ways: by increasing the grain size of the output program, by reducing synchronization and storage requirements, and by improving locality of reference. The two key Ideas behind these optimizations are the symbolic analysis of loop structures and hierarchical clustering of the program graph, first by loop structure and then by loop traversal patterns.A benchmark suite demonstrates both the efficiency of the output code and the effectiveness of the optimization.

The Advantages of Multiple Parallelizations in Combinatorial Search

“…The VCODE compiler for the Encore Multimax [2] and the NESL compiler for the Connection Machine CM-2 [I] support nested data parallelism, which can be used to express both types of parallelism in subgraph isomorphism. The Chores runtime system [8] and the parallelizing Fortran compiler for the iWarp [15] are recent examples of programming systems designed to support both data parallelism (e.g., parallel loops) and task (or functional) parallelism (e.g., tree search).…”

Efficient parallel algorithms for closest point problems.

“…Algorithms for closest point problems have a more dynamic and irregular flavor than the benchmark programs normally used to analyze machine performance. In addition, the algorithms are more sophisticated than some irregular benchmarks that are already in use [Cha91].…”

“…Blelloch and his students have shown that algorithms using these primitives can be efficiently implemented on SIMD and vector architectures [CBZ90]. In addition, Chatterjee has shown that with suitable compilers, programs based on these primitives can be efficiently translated to run on shared memory MIMD machines [Cha91].…”

“…Most of the code was ported from the runtime library used in Chatterjee's VCODE compiler for the Encore Multimax [Cha91]. The code was compiled without optimization because optimization introduced bugs into the parallel programs that were difficult to track down.…”

“…This overhead inhibits speedup somewhat with 32 threads, for example. Replacing the sequential routine with a parallel tree-sum would allow the scan algorithm to scale more gracefully [Cha91].…”