Performance‐based parallel loop self‐scheduling using hybrid OpenMP and MPI programming on multicore SMP clusters

Abstract

SUMMARYParallel loop self-scheduling on parallel and distributed systems has been a critical problem and it is becoming more difficult to deal with in the emerging heterogeneous cluster computing environments. In the past, some self-scheduling schemes have been proposed as applicable to heterogeneous cluster computing environments. In recent years, multicore computers have been widely included in cluster systems. However, previous researches into parallel loop self-scheduling did not consider certain aspects of multicore computers; for example, it is more appropriate for shared-memory multiprocessors to adopt Open Multi-Processing (OpenMP) for parallel programming. In this paper, we propose a performancebased approach using hybrid OpenMP and MPI parallel programming, which partition loop iterations according to the performance weighting of multicore nodes in a cluster. Because iterations assigned to one MPI process are processed in parallel by OpenMP threads run by the processor cores in the same computational node, the number of loop iterations allocated to one computational node at each scheduling step depends on the number of processor cores in that node. Experimental results show that the proposed approach performs better than previous schemes.

Load-prediction scheduling algorithm for computer simulation of electrocardiogram in hybrid environments

“…The advantage of static scheduling is that it is easy to implement and has no extra scheduling overhead during runtime, but it may cause load unbalancing and thereby reduce computational efficiency (Zhu et al, 2004;Plank et al, 2007). Dynamic scheduling adjusts the schedule during execution and is especially suitable for situations where the number of steps, or the workload of each step, is undetermined (Yang et al, 2011). Although dynamic scheduling is more suitable for load balancing among parallel processors, the scheduling overhead should be executed at runtime and it needs extra processing time.…”

“…This can be done by adjusting the chunk size. Therefore, these schemes cannot achieve load balancing in a CPU-GPU in an extremely heterogeneous environment (Yang et al, 2011).…”

“…Yang and Chang proposed a combinational approach to solve parallel regular loop scheduling problem in an extremely heterogeneous computer cluster environment (Yang et al, 2011). In their experiments, they assigned 80% workload corresponding to the CPU clock using static scheduling and 20% workload using dynamic scheduling.…”

An approach of performance comparisons with OpenMP and CUDA parallel programming on multicore systems

“…In our previous approaches [22,23,25,26], the implementation of automatic parallelization based on OpenMP currently supports both C and C++ codes. Based on the result of these researches, we enhanced and extended the research.…”

Knowledge-Based Adaptive Self-Scheduling

“…However, the size of each work queue is not determined based on the knowledge of loop and runtime environment in affinity scheduling. Some groups [9,10] have undertaken self-scheduling studies on particular architectures, considering the features of the system architecture. Our technique could be easily extended to these architectures.…”