Enabling CUDA acceleration within virtual machines using rCUDA

Abstract

The hardware and software advances of Graphics Processing Units (GPUs) have favored the development of GPGPU (General-Purpose Computation on GPUs) and its adoption in many scientific, engineering, and industrial areas. Thus, GPUs are increasingly being introduced in high-performance computing systems as well as in datacenters. On the other hand, virtualization technologies are also receiving rising interest in these domains, because of their many benefits on acquisition and maintenance savings. There are currently several works on GPU virtualization. However, there is no standard solution allowing access to GPGPU capabilities from virtual machine environments like, e.g., VMware, Xen, VirtualBox, or KVM. Such lack of a standard solution is delaying the integration of GPGPU into these domains. In this paper, we propose a first step towards a general and open source approach for using GPGPU features within VMs. In particular, we describe the use of rCUDA, a GPGPU (General-Purpose Computation on GPUs) virtualization framework, to permit the execution of GPU-accelerated applications within virtual machines (VMs), thus enabling GPGPU capabilities on any virtualized environment. Our experiments with rCUDA in the context of KVM and VirtualBox on a system equipped with two NVIDIA GeForce 9800 GX2 cards illustrate the overhead introduced by the rCUDA middleware and prove the feasibility and scalability of this general virtualizing solution. Experimental results show that the overhead is proportional to the dataset size, while the scalability is similar to that of the native environment.

CU2rCU: Towards the complete rCUDA remote GPU virtualization and sharing solution

“…As our rCUDA virtualization solution aims at being compatible with the latest release, it must evolve to support the new CUDA versions. In this regard, the work presented in [5], [6], [7] supported the now obsolete CUDA 2 and 3 versions. After those initial versions of rCUDA, NVIDIA released CUDA 4, with significant changes with respect to prior versions.…”

“…Previous work in [5], [6], [7] mainly focused on demonstrating that using remote CUDA devices is feasible. Nevertheless, three main concerns quickly arose during the completion of those studies:…”

“…In order to enable our disruptive power-efficient proposal for HPC deployments, we have recently developed the rCUDA framework [5], [6], [7]. Our technology employs a clientserver middleware.…”

FairGV: Fair and Fast GPU Virtualization

“…Most of the aforementioned solutions provide a shared memory mechanism for communication between the guest and the host. An exception is rCUDA, which aims at utilizing remote GPUs and uses TCP/IP-based communication for both local and remote GPU virtualization [17].…”

“…GVirtuS with its shared memory module can suffer from performance degradation when the kernel size is under 250 µs (CFD, HS, SRD1, MM, and SCAN). Trapping to the OS kernel per request causes high overhead as explained in Section 3.1. rCUDA uses TCP/IP for inter-VM communication [17]. For high network performance, we enabled virtio, which is a para-virtualized network driver for KVM and offers up to 30 Gbps of inter-VM bandwidth in our system.…”