Integrated Support for Task and Data Parallelism

Abstract

We present an overview of research at the Center for Research on Parallel Computation designed to provide an efficient, portable programming model for scientific applications possessing both task and data parallelism. Fortran M programs exploit task parallelism by provid ing language extensions for user-defined process man agement and typed communication channels. A com bination of compiler and run-time system support en sures modularity, safety, portability, and efficiency. Fortran D and High Performance Fortran programs ex ploit data parallelism by providing language exten sions for user-defined data decomposition specifica tions, parallel loops, and parallel-array operations.

Dynamic Data Distribution and Processor Repartitioning for Irregularly Structured Computations

“…We run both versions on each of the partition sizes we consider in this study (4,8,16, and 32 PEs) and prevented any reconfigurations from occurring during the execution of the DRMS versions. Therefore, in all cases the factorization runs from beginning to end on the same number of processors.…”

“…Finally, we note that our research is related to work that combines simultaneous exploitation of task and data parallelism in the same application, such as those described in [4] and [29]. The issue of rearranging processors among competing applications is similar to that of subdividing a processor partition among concurrent tasks of the same application.…”

“…2b). Note that the range of valid partition sizes has changed to (4,8,16,32). The partition size can change between S 1 and the first instance of S 2 and also between different instances of S 2 .…”

“…In this particular case, the code specifies a range consisting of all numbers between 4 and 64 that are squares of integers. Therefore, valid partition sizes for the computation of the factors (stage S 1 ) are (4,9,16,25,36,49,64). DRMS also allows the specification of ranges with additive step, multiplicative step, and irregular ranges specified in the form of lists.…”

Integrated Task and Data Parallel Support for Dynamic Applications

“…Unfortunately, existing techniques for integrated task and data parallel computing [2,4,9,19,3] are not well-suited to this type of application. As an example of the current state-of-the-art, the system described in [19] employs extensive off-line analysis of the memory requirements, communication patterns and scalability aspects of the tasks.…”