Unique Sets Oriented Parallelization of Loops with Non-uniform Dependences

Abstract

Although many methods exist for nested loop partitioning, most of them perform poorly when parallelizing loops with non-uniform dependences. This paper addresses the issue of automatic parallelization of loops with non-uniform dependences. Such loops are normally not parallelized by existing parallelizing compilers and transformations. Even when parallelized in rare instances, the performance is very poor. Our approach is based on the`convex hull' theory which has adequate information to handle non-uniform dependences. We introduce the concept of`complete dependence convex hull',`unique head and tail sets' and abstract the dependence information into these sets. These sets form the basis of the iteration space partitions. The properties of the unique head and tail sets are derived. Depending on the relative placement of these unique sets, partitioning schemes are suggested for implementation of our technique. Implementation results of our scheme on the Cray J916 and comparison with other schemes show the superiority of our technique.

Non-uniform dependences partitioned by recurrence chains

“…A number of alternatives have been proposed for the case of affine index expressions, e.g. uniformization oriented techniques [23,26,6,19,27] and dataflow oriented techniques [20,11]. In this paper, the loops with non-uniform dependences are parallelized using WHILE loops with irregular strides.…”

“…Since det(T) = 3, the largest partition has at most 1 + log 3 ( N 2 1 + N 2 2 ) iterations by theorem 1. Example 2 Consider another non-uniform dependence example used by Ju et al [11]. The PDM partitioning can only find a parallelism of two in the innermost loop, thus the recurrence chain partitioning is applied using algorithm 1: a(2*i+3,j+1)=a(i+2*j+1,i+j+3) 5 ENDDOALL 6 ENDIF 7 DOALL j=(i+2)/2,min(i+3,-i+10) 8 a(2*i+3,j+1)=a(i+2*j+1,i+j+3) 9 ENDDOALL 10 DOALL j=max(-i+11,1),min(i+3,12) 11 a(2*i+3,j+1)=a(i+2*j+1,i+j+3) 12 ENDDOALL 13 DOALL j=(3*i+8)/2,12 14 a(2*i+3,j+1)=a(i+2* (2,6).…”

“…However, it drops below linear when number of threads is larger than 3 because the loop bounds calculation gets more overhead. For Example 2 with parameter N=300, the UNIQUE [11] and REC speedups are compared. They both outperforms linear speed when executed on single CPU because the convex loop index calculations are optimized by Omega calculator.…”

“…Theoretically, it speedups Example 2 by 4 times. Ju et al [11] propose unique-set oriented partitioning to exploit exact non-uniform dependences: The dependence convex hulls are separated into head or tail sets by lexicographical order. The first recurrence equation is called "flow" and the second is called "anti", which split the head or tail sets.…”

“…To avoid introducing artificial dependences into the loop, another way of treating the non-uniform dependences is based on exact solution of the dependence equations [11]. Although it is generally impossible to enforce data flow scheduling at compile time, recurrence chain partitioning makes it possible when there is only one pair of coupled subscripts or there is no compile-time unknown variable in the loop bounds.…”