POSTER: Boosting the performance of remote GPU virtualization using InfiniBand connect-IB and PCIe 3.0

Abstract

Abstract-A clear trend has emerged involving the acceleration of scientific applications by using GPUs. However, the capabilities of these devices are still generally underutilized. Remote GPU virtualization techniques can help increase GPU utilization rates, while reducing acquisition and maintenance costs. The overhead of using a remote GPU instead of a local one is introduced mainly by the difference in performance between the internode network and the intranode PCIe link. In this paper we show how using the new InfiniBand Connect-IB network adapters (attaining similar throughput to that of the most recently emerged GPUs) boosts the performance of remote GPU virtualization, reducing the overhead to a mere 0.19% in the application tested.

On the Execution of Computationally Intensive CPU-Based Libraries on Remote Accelerators for Increasing Performance: Early Experience with the OpenBLAS and FFTW Libraries

“…For example, the InfiniBand Verbs API may be used instead of the TCP/IP protocol stack, boosting network throughput. Also, an efficient communication pipeline could be leveraged, as in the rCUDA remote GPU virtualization framework [10]. Another possibility is using the GPU Direct RDMA mechanism provided by NVIDIA and Mellanox [17].…”

“…The performance estimation methodology consists in replacing, in the results presented in the previous section, the communication time between main memory in the client and the GPU memory in the server (including the intermediate stop at the server's main memory) by the time that an optimized communication layer would attain. Notice that for estimating the time required to move data to and from the remote server, which depends on the volume of input and output data and also on the network bandwidth attained for each transfer size, the bandwidth achieved by the rCUDA remote GPU virtualization framework [10] has been used instead of using the raw bandwidth of the network fabric. This approach is more accurate than using the raw InfiniBand bandwidth because software layers always impose some loss to theoretical performance numbers.…”

“…This is the case of networked disks, which allow a file system to be shared among many different computers. Likewise, it is possible to provide GPU-acceleration services to a cluster by sharing a networked GPU by means of the remote GPU virtualization technique, which has been implemented in frameworks such as rCUDA [10], GVirtuS [11], or DS-CUDA [12], among others. Furthermore, when remote GPU virtualization solutions are considered at the cluster level, they provide noticeable reductions in the total execution time of a given workload composed of a set of computing jobs [13] and in the total energy required to execute such workloads [14].…”

On the Virtualization of CUDA Based GPU Remoting on ARM and X86 Machines in the GVirtuS Framework

“…GPU virtualization solutions to GPGPU as GVirtuS have been implemented in research projects as rCUDA (Remote CUDA) [33] e DS-CUDA (DistributedShared CUDA) [17]. They all use an approach similar to GVirtuS, providing CUDA API wrappers on the front-end application in the guest OS while the back-end in the host OS accesses to the CUDA devices.…”