Hierarchically tiled arrays for parallelism and locality

Abstract

Parallel programming is facilitated by constructs which, unlike the widely used SPMD paradigm, provide programmers with a global view of the code and data structures. These constructs could be compiler directives containing information about data and task distribution, language extensions specifically designed for parallel computation, or classes that encapsulate parallelism. In this paper, we describe a class developed at Illinois and its MATLAB implementation. This class can be used to conveniently express both parallelism and locality. A C++ implementation is now underway. Its characteristics will be reported in a future paper. We have implemented most of the NAS benchmarks using our HTA MATLAB extensions and found during that HTAs enable the fast prototyping of parallel algorithms and produce programs that are easy to understand and maintain.

Swizzle Inventor

“…These functions essentially provide an assignment of data to threads similar to existing frameworks that support decomposing arrays and processing them using different levels of the compute hierarchy. These include Chapel [8], Hierarchically Tiled Arrays (HTA) [9] and Lift [10]. However, programmers still have to supply the shfl argument manually in these frameworks.…”

“…Another kind of permutations that our current template does not cover is moving u consecutive elements together as a unit when u divides n. To cover this kind of permutations, we can introduce a unit primitive: unit(n, u, p)(i) = p(⌊n/u⌋)(⌊i/u⌋) * u + (i mod u) (9) Similar to дroup, it takes a permutation p as an argument.…”