Efficient Inter-Device Task Scheduling Schemes for Multi-Device Co-Processing of Data-Parallel Kernels on Heterogeneous Systems

Abstract

Heterogeneous systems consisting of multiple multi-core CPUs and many-core accelerators have recently come into wide use, and more and more parallel applications are developed in such a heterogeneous system. To fully utilize multiple compute devices to cooperatively and concurrently execute data-parallel kernels on heterogeneous systems, a feedback-based dynamic and elastic task scheduling scheme is proposed, which can provide a better load balance, a greater device utilization, and a lower scheduling overhead by flexibly and dynamically adjusting the workload between devices during execution. The proposed method is more suitable for data-parallel kernels whose computation and data are uniformly distributed, but is less suitable for data-parallel kernels whose computation and data are non-uniformly distributed. Thus, an asynchronous-based dynamic and elastic task scheduling scheme is proposed, which can avoid device underutilization, load imbalance across devices, and frequent kernel launches, interdevice data transfers and inter-device synchronizations by dynamically adjusting the chunk size according to the performance change during runtime. A series of experiments are conducted with 8 representative parallel applications on a hybrid CPU-GPU-MIC system, the results show that the proposed two interdevice task scheduling schemes can achieve the efficient CPU-GPU-MIC co-processing of different parallel applications by effectively partitioning work across devices.INDEX TERMS Data-parallel kernels, heterogeneous systems, many-core accelerators, multi-core CPUs, multi-device co-processing, parallel applications, task scheduling.

HeteroPP: A directive‐based heterogeneous cooperative parallel programming framework

“…For the sake of brevity, the details of HCE runtime system and inter-device scheduling policies will not be described in this article, please refer to our previous work. 17,18 In short, under the help of HeteroPP, programmers only need to focus on how to write data-parallel compute kernels using the extended OpenMP directives and clauses, but do not need to care about the complicated implementation details of multi-device co-computing of data-parallel compute kernels.…”

“…The runtime system is mainly composed of four components: device management, memory management, task scheduling, and transfer optimization, and each component contains a series of runtime application programming interfaces (APIs). The runtime system currently provides our previously proposed inter‐device scheduling policies 17,18 including FDETS (i.e., the feedback‐based dynamic and elastic task scheduling policy), MFDETS (i.e., the modified FDETS that supports incremental data transfer), ADETS (i.e., the asynchronous‐based dynamic and elastic task scheduling policy), and MADETS (i.e., the modified ADETS that supports three‐way overlapping communication optimization). For the sake of brevity, the details of HCE runtime system and inter‐device scheduling policies will not be described in this article, please refer to our previous work 17,18 …”

“…As shown in Figure 4, programmers can specify matrix A should be partially copied from host to the GPU and MIC by using the following map clause: map ( part to : A [0: $$$ M\ast L $$$: $$$ L $$$]). In fact, the data distribution across computing devices is automatically addressed by our previously proposed task scheduling policies, 17,18 which can dynamically identify what data needs to be copied to or from which accelerator and when.…”

“…In this paper, on the basis of our previously proposed inter‐device task scheduling policies 17 and runtime system 18 that can efficiently support heterogeneous cooperative execution, we propose HeteroPP to provide an easier way to efficiently support the multi‐device co‐computing of data‐parallel compute kernels, which can fully exploit multiple computing devices of the same or different types to concurrently and cooperatively perform data‐parallel compute kernels on heterogeneous platforms. HeteroPP provides a high‐level abstraction of heterogeneous cooperative parallel programming by extending OpenMP.…”

Graph Processing Scheme Using GPU With Value-Driven Differential Scheduling

“…In SIMD computing, several values are processed simultaneously using a single instruction, contrasting with the typical structure of CPUs. Consequently, GPUs necessitate specific data structure and scheduling approaches to fully exploit their parallel capabilities [12,[18][19][20][21][22][23]. Owing to their relatively smaller memory than the Host, GPUs can only accommodate a subset of the complete graph as input.…”