A general approach for partitioning N-dimensional parallel nested loops with conditionals

Abstract

Parallel loops account for the greatest amount of parallelism in scientific and numerical codes. For example, most of the DO loops in SPEC CFP2000 and SPEC OMPM2001 are of DOALL type and account for a large percentage of the total execution time. One of the ways to exploit parallelism is to partition the iteration space of a DOALL loop amongst different processors in a parallel processor system. Naturally, a good partitioning is of key importance to achieve high performance and for efficient use of multiprocessor systems. Although a significant amount of work has been done in partitioning and scheduling of loops with both rectangular and non-rectangular iteration spaces, the problem of partitioning loops with conditionals has not been addressed so far to the best of our knowledge. In this paper, we present a mathematical model for partitioning parallel nested loops, both perfect and non-perfect, with conditionals, where the expressions in a conditional are affine functions of the outer loop indices. We present a loop transformation based on elimination of redundant constraints bounding the iteration space of a nested loop. The transformation plays a critical role during the (static) partitioning process as it helps to capture the "exact" lower and upper bounds (can be either a constant or symbolic) of the loop indices. We generate a canonical form of the loop nest using the transformation and employ the geometric approach we proposed earlier (in [1, 2]) for partitioning the iteration space along an axis corresponding to the outermost loop. For cases in which such a transformation does not exist, we propose a general approach for loop canonicalization. We present several examples from the literature and numerical packages to illustrate the effectiveness of our approach.

pocl: A Performance-Portable OpenCL Implementation

“…For example, improving the parallelization of kernels with diverging branches (parts executed only by a subset of the work-items) is one of the low-hanging fruits. There is some previous work available that is targeted towards enhanced load-balancing which could be adapted to improving the fine-grained parallelization on machines with limited support for predication as well [32]. 1 7 .…”

Cache-aware iteration space partitioning

“…Unlike [8], the non-perfect nature of the loop model is not restricted to conditionals. The loop model also supports multi-way loops [14], i.e., multiple loops may be present at the same level.…”

“…A significant amount of work has been done in the context of static partitioning of (parallel) loop nests with rectangular as well as non-rectangular iteration spaces [4,5,6,7,8,9,10]. However, the existing techniques are cache-oblivious, i.e., they do not capture the variation in the number of cache misses across the iteration space.…”