Stencil Calculations with Algorithmic Skeletons for Heterogeneous Computing Environments

Abstract

The development of parallel applications is a difficult and error-prone task, especially for inexperienced programmers. Stencil operations are exceptionally complex for parallelization as synchronization and communication between the individual processes and threads are necessary. It gets even more difficult to efficiently distribute the computations and efficiently implement communication when heterogeneous computing environments are used. For using multiple nodes, each having multiple cores and accelerators such as GPUs, skills in combining frameworks such as MPI, OpenMP, and CUDA are required. The complexity of parallelizing the stencil operation increases the need for abstracting from the platform-specific details and simplify parallel programming. One way to abstract from details of parallel programming is to use algorithmic skeletons. This work introduces an implementation of the MapStencil skeleton that is able to generate parallel code for distributed memory environments, using multiple nodes with multicore CPUs and GPUs. Examples of practical applications of the MapStencil skeleton are the Jacobi Solver or the Canny Edge Detector. The main contribution of this paper is a discussion of the difficulties when implementing a universal Skeleton for MapStencil for heterogeneous computing environments and an outline of the identified best practices for communication intense skeletons.

Reusable Algorithmic Skeleton Framework for Clustering Algorithms in Wireless Sensor Network (SCW)

“…Due to the advantages of the algorithmic skeleton, researchers utilize it in different fields of computer engineering science. Some examples are listed in the following, accompanied by a few article samples: Parallel programming for distributed memory environments [16], Mainstream parallel programming [17], Multi-core clusters [18], Evolutionary multi-agent systems in Erlang [19], [20], High-performance computing [21], [22], Heterogeneous CPU-GPU architectures [23]- [25], Using MapReduce model in Big data [26], Presenting fast solver for structured linear systems [27], Evaluating approximate computing and heterogeneity for energy efficiency [28], Replicable parallel branch and bound search [29] and Wireless sensor networks [30], [31].…”

EPSILOD: efficient parallel skeleton for generic iterative stencil computations in distributed GPUs

“…There are other works oriented toward executing large stencil computations for distributed systems using GPU coprocessors, some of them based on parallel skeletons, such as [33][34][35][36][37][38][39][40][41][42]. These works achieve a better performance due to the use of this kind of hardware accelerators.…”

“…These works achieve a better performance due to the use of this kind of hardware accelerators. In particular, Muesli [42] and SkelCL [39] are good examples of skeleton-based solutions that fall in this category. However, they do not implement a full overlapping of data communication and computation, combining the overlapping opportunities in transfers between host and devices with communication and synchronization across nodes.…”