Dynamic Loop Scheduling Using MPI Passive-Target Remote Memory Access

Abstract

Scientific applications often contain large computationally-intensive parallel loops. Loop scheduling techniques aim to achieve load balanced executions of such applications. For distributed-memory systems, existing dynamic loop scheduling (DLS) libraries are typically MPI-based, and employ a master-worker execution model to assign variably-sized chunks of loop iterations. The master-worker execution model may adversely impact performance due to the master-level contention. This work proposes a distributed chunk-calculation approach that does not require the master-worker execution scheme. Moreover, it considers the novel features in the latest MPI standards, such as passive-target remote memory access, shared-memory window creation, and atomic read-modify-write operations. To evaluate the proposed approach, five well-known DLS techniques, two applications, and two heterogeneous hardware setups have been considered. The DLS techniques implemented using the proposed approach outperformed their counterparts implemented using the traditional master-worker execution model.

Hierarchical Dynamic Loop Self-Scheduling on Distributed-Memory Systems Using an MPI+MPI Approach

“…Implementation Approaches: Hierarchical DLS techniques can be implemented either using the hierarchical master-worker [13] or using the distributed chunk-calculation model [15]. The present work evaluates the use of two different implementations, MPI+OpenMP and MPI+MPI, to complement the distributed chunk-calculation approach (see Section 2).…”

“…Recently, a distributed chunk-calculation approach was proposed for developing DLS techniques executing on distributed-memory systems [15]. This approach eliminated the use of the master-worker model by exploiting the one-sided communication features offered in the MPI-3 standard.…”

“…The proposed approach applies two DLS techniques at the intra-and inter-node levels as follows: one MPI process creates a global shared-memory region, called global work queue. This global queue stores information regarding the latest scheduling step and the total scheduled loop iterations [15]. Using MPI Win allocate shared, the MPI processes within one compute node create another shared-memory region, called local work queue.…”

“…This work proposes a novel approach for designing and developing hierarchical DLS techniques for distributed-memory systems. It extends the distributed chunk calculation approach [15] by allowing any group of workers to reside on a shared-memory system to form a shared work queue where the chunks to be executed by this group are stored. The novelty of the proposed approach lies in the fact that the responsibility of the work queue is shared among the workers of the group.…”

A Distributed Chunk Calculation Approach for Self-scheduling of Parallel Applications on Distributed-memory Systems

“…Although centralizing the chunk assignment does not mean centralizing the chunk calculation, many of the recent DLS techniques employ a master-worker execution model that centralizes both the chunk calculation and the chunk assignment at the master side [6,7,8,9,10]. The current work extends our earlier distributed chunk calculation approach (DCA) [11] and makes the following unique contributions.…”