Generalizing the Utility of GPUs in Large-Scale Heterogeneous Computing Systems

Abstract

Abstract-Graphics processing units (GPUs) have been widely used as accelerators in large-scale heterogeneous computing systems. However, current programming models can only support the utilization of local GPUs. When using non-local GPUs, programmers need to explicitly call API functions for data communication across computing nodes. As such, programming GPUs in large-scale computing systems is more challenging than local GPUs since local and remote GPUs have to be dealt with separately. In this work, we propose a virtual OpenCL (VOCL) framework to support the transparent virtualization of GPUs. This framework, based on the OpenCL programming model, exposes physical GPUs as decoupled virtual resources that can be transparently managed independent of the application execution. To reduce the virtualization overhead, we optimize the GPU memory accesses and kernel launches. We also extend the VOCL framework to support live task migration across physical GPUs to achieve load balance and/or quick system maintenance. Our experiment results indicate that VOCL can greatly simplify the task of programming cluster-based GPUs at a reasonable virtualization cost.Keywords-graphics processing unit (GPU), virtual OpenCL, task migration I. INTRODUCTION Today, the growth rate of computational power required for some scientific computations has been outstripping that of the computational capabilities of traditional processors. On the other hand, gains in computational horsepower by increasing the clock speed have hit a wall in the mid-2000s due to the fabrication process and power/leakage constraints. As a consequence, parallelism must be relied upon in order to improve computational performance. At the hardware level, performance improvements have been driven by increasing the number of cores per processor, which is true for both traditional x86 multicore processors and many-core GPU processors. These multi-and many-core processors in turn have been aggregated into compute nodes, which are then aggregated into cluster supercomputers.Originally designed solely for graphics processing, GPUs have evolved into programmable processors for generalpurpose computation. With parallel programming models such as CUDA [10] and OpenCL [7]), programming GPUs has become easier than ever. Many applications have now been parallelized on GPUs, and significant performance improvements have been reported [11].However, current GPU programming models only support the utilization of GPUs installed locally. When a program needs to use GPUs that are not locally installed, API functions such as TCP sockets and Message Passing Interface (MPI) [8] should be called explicitly to handle data communication

Heterogeneous distributed computing based on high‐level abstractions

“…However, interesting efforts to program clusters of Xeon Phi only based on compiler directives such as OmpSs, discussed later, have also been made . A more ambitious approach is to enable the execution of unaltered or very slightly modified heterogeneous applications written using well‐known frameworks such as CUDA or OpenCL on distributed systems, so that they can exploit remote accelerators in clusters, grids, and the cloud, typically by virtualizing them. Since the main purpose of these proposals is not to provide higher level semantics for the programming of the distribution resources but to simplify their exploitation as much as possible in application developed using CUDA or OpenCL; most of these tools expose abstractions, and thus APIs that are at the low level of these tools, being in fact nearly identical in most cases.…”

Exascale Machines Require New Programming Paradigms and Runtimes

“…OpenCL Remote [58] is a framework which extends both OpenCL's platform model and memory model with a network client-server paradigm. Virtual OpenCL [57], based on the OpenCL programming model, exposes physical GPUs as decoupled virtual resources that can be transparently managed independent of the application execution.…”

LibWater

“…In recent years, heterogeneous systems have received a great amount of attention from the research community. Although several projects have been recently proposed to facilitate the programming of clusters with heterogeneous nodes [20,8,6,5,18,12,26,31], none of them combines support for high performance inter-node data transfer, support for a wide number of different devices and a simplified programming model. Our work takes into account all this aspects through the development of the libWater library.…”

“…OpenCL Remote [26] is a framework which extends both OpenCL's platform model and memory model with a network client-server paradigm. Virtual OpenCL [31], based on the OpenCL programming model, exposes physical GPUs as decoupled virtual resources that can be transparently managed independent of the application execution.…”