New OpenMP Directives for Irregular Data Access Loops

Abstract

Many scientific applications involve array operations that are sparse in nature, ie array elements depend on the values of relatively few elements of the same or another array. When parallelised in the shared-memory model, there are often inter-thread dependencies which require that the individual array updates are protected in some way. Possible strategies include protecting all the updates, or having each thread compute local temporary results which are then combined globally across threads. However, for the extremely common situation of sparse array access, neither of these approaches is particularly efficient. The key point is that data access patterns usually remain constant for a long time, so it is possible to use an inspector/executor approach. When the sparse operation is first encountered, the access pattern is inspected to identify those updates which have potential inter-thread dependencies. Whenever the code is actually executed, only these selected updates are protected. We propose a new OpenMP clause, indirect, for parallel loops that have irregular data access patterns. This is trivial to implement in a conforming way by protecting every array update, but also allows for an inspector/executor compiler implementation which will be more efficient in sparse cases. We describe efficient compiler implementation strategies for the new directive. We also present timings from the kernels of a Discrete Element Modelling application and a Finite Element code where the inspector/executor approach is used. The results demonstrate that the method can be extremely efficient in practice.

Workload Decomposition Strategies for Shared Memory Parallel Systems with OpenMP

“…Extensions to OpenMP able to handle irregular array reductions have been proposed by J. Labarta et al [14], but neither the present version of OpenMP nor the upcoming one (2.0) present such capability.…”

“…[14], the pressure computation can be (very simply) recoded as follows: p = 0. !$OMP parallel do private(l,j r, j theta,j phi) !$OMP& reduction(+:p) !$OMP& indirect(j r,j theta,j phi) do l = 1,n part j r = f r(r(l)) j theta = f theta(theta(l)) j phi = f phi(phi(l)) p(j r,j theta,j phi) = p(j r, j theta,j phi) & + h(r(l),.…”

“…[14]), where the elements to be reduced are the particle coordinates (the elements of the arrays r, theta, phi), and the results of the reduction are the pressure values (the elements of the array p). The operation is a reduction because the updating function h has associative and distributive properties with respect to the contributions given by every single particle (i.e.…”

Workload decomposition strategies for hierarchical distributed‐shared memory parallel systems and their implementation with integration of high‐level parallel languages

“…Such a computation is indeed an example of an irregular arrayreduction operation (cf., e.g., [13]), where the elements to be reduced are the particle coordinates (the elements of the arrays r, theta, phi), and the results of the reduction are the pressure values (the elements of the array p par). The operation is a reduction because the updating function h has associative and distributive properties with respect to the contributions given by every single particle (i.e.…”

A Performance‐Prediction Model for PIC Applications on Clusters of Symmetric MultiProcessors: Validation with Hierarchical HPF+OpenMP Implementation

“…The immediate intra-node parallelization of the pressure loop is however inhibited, as in the inter-node case, by the updating of the array p par. Such a computation is indeed an example of irregular array-reduction operation (cf., e.g., [9]), where the elements to be reduced are the particle coordinates (the elements of the arrays r, theta, phi), and the results of the reduction are the pressure values (the elements of the array p par). The operation is a reduction because the updating function h has associative and distributive properties with respect to the contributions given by every single particle (i.e.…”