A Library-Based Approach to Task Parallelism in a Data-Parallel Language

Abstract

The data-parallel language High Performance Fortran (HPF) does not allow e cient expression of mixed task/data-parallel computations or the coupling of separately compiled data-parallel modules. In this paper, we show how these common parallel program structures can be represented, with only minor extensions to the HPF model, by using a coordination library based on the Message Passing Interface (MPI). This library allows data-parallel tasks to exchange distributed data structures using calls to simple communication functions. We present microbenchmark results that characterize the performance of this library and that quantify the impact of optimizations that allow reuse of communication schedules in common situations. In addition, results from two-dimensional FFT, convolution, and multiblock programs demonstrate that the HPF/MPI library can provide performance superior to that of pure HPF. We conclude that this synergistic combination of two parallel programming standards represents a useful approach to task parallelism in a data-parallel framework, increasing the range of problems addressable in HPF without requiring complex compiler technology.

Evaluating Computational Costs While Handling Data and Control Parallelism

“…Tab. 4 shows the evaluation process of P ipe(it f , it d ) labelling each transformation with its cost load.…”

“…On the other hand, a parallel application have also to deal with data access concerns that can heavily influence both the programming phase and the final computational costs. Unfortunately, the most works regarding structured parallel programming environments, define a two-tier architecture model in which data accesses have to be heavily detailed on top of control abstractions [2,3] or vice versa [4,5,6,7]. As a consequence, the model of parallelism is a mixture of strictly coupled data and control concerns difficult to formalize and, hence, to statically and/or dynamically analyze for optimization purpose.…”

SBASCO: skeleton-based scientific components

“…The fft 2d component of the example described above, can be structured as a pipeline of two stages in order to increase its efficiency, as proved in different works [11] [5]. The first one performs independent one-dimensional FFTs on the columns of the input arrays.…”

Advanced parallel programming models research and development opportunities.

“…This model is typically used on clusters of shared memory machines, where a shared memory model is used within a node and a message passing model is used to exchange data between nodes. Examples of hybrid models include OpenMP with MPI [23] and HPF with MPI [20].…”