Efficient Shared Memory Message Passing for Inter-VM Communications

Abstract

Abstract. Thanks to recent advances in virtualization technologies, it is now possible to benefit from the flexibility brought by virtual machines at little cost in terms of CPU performance. However on HPC clusters some overheads remain which prevent widespread usage of virtualization. In this article, we tackle the issue of inter-VM MPI communications when VMs are located on the same physical machine. To achieve this we introduce a virtual device which provides a simple message passing API to the guest OS. This interface can then be used to implement an efficient MPI library for virtual machines. The use of a virtual device makes our solution easily portable across multiple guest operating systems since it only requires a small driver to be written for this device. We present an implementation based on Linux, the KVM hypervisor and Qemu as its userspace device emulator. Our implementation achieves near native performance in terms of MPI latency and bandwidth.

Characterizing the Performance of Parallel Applications on Multi-socket Virtual Machines

“…Although we use the virtio driver which generally achieves about 70% of the native hardware bandwidth, the performance degradation is large (up to 16 times) when increasing the number of VMs. Many other [28], [12], [19], [5] research activities tried to address communication problems in virtualized environments but we did not find any mature and usable solution for efficient communication between VMs using loopback.…”

“…In contrast, Xen-based virtualization [4] allows sharing pages between only two VMs [28], [5], [12] using the Grant- Table and it imposes severe restrictions on the amount of memory allowed. We have also modified Xen to provide multi-VM sharing and increase the amount of memory shared.…”

Residency-Aware Virtual Machine Communication Optimization: Design Choices and Techniques

“…This is because the abstraction of VMs supported by VMM technology does not differentiate whether the data request is coming from co-resident VMs or not. Several research projects [2][3][4][5][6][7][8][9][10][11] have demonstrated that. Linux guest domain shows lower network performance than native Linux [12], when an application running on a VM communicates with another VM.…”

“…Related work such as IVC [2], XenSocket [3], XWAY [4], XenLoop [5], MMNet [6] above Fido [7] and XenVMC [8] are on Xen platform. While all KVM-based efforts, such as VMPI [9], Socket-outsourcing [10] and Nahanni [11] In this paper, we provide an in-depth overview of the design choices and techniques for optimizing the performance of the co-resident inter-VM communication, with dual objectives. First, we describe the core design guidelines and key issues for optimizing inter-VM communication using shared memory based mechanisms.…”

Xen2MX: High-performance communication in virtualized environments

“…Unfortunately, in the cloud context, I/O-intensive applications suffer from poor performance [22,33,40], due to various intermediate layers that abstract away the physical characteristics of the underlying hardware and multiplex the application's access to I/O resources. This limitation is one of the most important reasons that HPC applications are not widely deployed in virtualized environments [35].Numerous studies both in native [8,16] and virtualized environments [5,6,18,20,22,23,24,25,26,40] explore the implications of alternative data-paths that increase the system's I/O throughput, helping applications overcome significant bottlenecks in data retrieval from storage or network devices. However, near-native I/O performance for Virtual Machines (VMs) in a generic cloud environment, built from offthe-shelf components, is still far from being achieved.…”