Bridging the Virtualization Performance Gap for HPC Using SR-IOV for InfiniBand

Musleh, Malek; Pai, Vijay S.; Walters, John Paul; Younge, Andrew J; Crago, Stephen P.

doi:10.1109/cloud.2014.89

Cited by 18 publications

(8 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The use of an IB interconnect for building HPC clouds has recently gained attention in the HPC community [10], [30]. In particular, advancements in hardware virtualization techniques, such as single root I/O virtualization (SR-IOV), have opened doors to efficient execution of HPC applications in the cloud [11], [31], [32]. More recently, containerized execution of HPC applications has also been studied [33].…”

Section: Related Workmentioning

confidence: 99%

A Self-Adaptive Network for HPC Clouds: Architecture, Framework, and Implementation

Zahid

Taherkordi

Gran

et al. 2018

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

Clouds offer flexible and economically attractive compute and storage solutions for enterprises. However, the effectiveness of cloud computing for high-performance computing (HPC) systems still remains questionable. When clouds are deployed on lossless interconnection networks, like InfiniBand (IB), challenges related to load-balancing, low-overhead virtualization, and performance isolation hinder full potential utilization of the underlying interconnect. Moreover, cloud data centers incorporate a highly dynamic environment rendering static network reconfigurations, typically used in IB systems, infeasible. In this paper, we present a framework for a self-adaptive network architecture for HPC clouds based on lossless interconnection networks, demonstrated by means of our implemented IB prototype. Our solution, based on a feedback control and optimization loop, enables the lossless HPC network to dynamically adapt to the varying traffic patterns, current resource availability, workload distributions, and also in accordance with the service provider-defined policies. Furthermore, we present IBAdapt, a simplified ruled-based language for the service providers to specify adaptation strategies used by the framework. Our developed self-adaptive IB network prototype is demonstrated using state-of-the-art industry software. The results obtained on a test cluster demonstrate the feasibility and effectiveness of the framework when it comes to improving Quality-of-Service compliance in HPC clouds.

show abstract

Section: Related Workmentioning

confidence: 99%

A Self-Adaptive Network for HPC Clouds: Architecture, Framework, and Implementation

Zahid

Taherkordi

Gran

et al. 2018

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

show abstract

“…While the Aries network provides considerable bandwidth and latency advantages over InfiniBand or Ethernet, it does not support Single Root I/O Virtualization (SR-IOV) as often found with newer Mellanox-based interconnects. Using direct PCI passthrough of the Aries NIC, like initial InfiniBand efforts [36], is also not possible as it would rob the Host CNL OS from any network. Furthermore, the Cray device drivers necessary are not be available within a commodity guest OS.…”

Section: Guest Performance Tuningmentioning

confidence: 99%

Enabling Diverse Software Stacks on Supercomputers Using High Performance Virtual Clusters

Younge

Pedretti

Grant

et al. 2017

2017 IEEE International Conference on Cluster Computing (CLUSTER)

View full text Add to dashboard Cite

While large-scale simulations have been the hallmark of the High Performance Computing (HPC) community for decades, Large Scale Data Analytics (LSDA) workloads are gaining attention within the scientific community not only as a processing component to large HPC simulations, but also as standalone scientific tools for knowledge discovery. With the path towards Exascale, new HPC runtime systems are also emerging in a way that differs from classical distributed computing models. However, system software for such capabilities on the latest extreme-scale DOE supercomputing needs to be enhanced to more appropriately support these types of emerging software ecosystems.

show abstract

“…Various studies have been performed on virtualization of RDMA devices, covering both hardware based solutions with the support of PCI SR-IOV [4,9], and also para-virtualization solutions [2,12,14,11,7]. Most of the previous work in this field was focused on the InfiniBand architecture [3], followed by RoCE and iWARP.…”

Section: Related Workmentioning

confidence: 99%

Lightweight and Generic RDMA Engine Para-Virtualization for the KVM Hypervisor

Mouzakitis

Pinto

Nikolaev

et al. 2017

2017 International Conference on High Performance Computing &Amp; Simulation (HPCS)

View full text Add to dashboard Cite

Remote DMA (RDMA) engines are widely used in clusters/data-centres to improve the performance of data transfers between applications running on different nodes of a computing system. RDMAs are today supported by most network architectures and distributed programming models. However, with the massive usage of virtualization most applications will use RDMAs from virtual machines, and the virtualization of such I/O devices poses several challenges. This paper describes a generic para-virtualization framework based on API Remoting, providing at the same time the flexibility of software based virtualization, and the low overhead of hardware-assisted solutions.The solution presented in this paper is targeting the KVM hypervisor, but is not bound to any target network architecture or specific RDMA engine, thanks to the virtualization at the level of the programming API. In addition, two of the major limitations of para-virtualization are addressed: data sharing between host and guest, and interactions between guests and hypervisor. A set of experimental results showed a near to native performance for the final user of the RDMA (i.e., maximum transfer bandwidth), with a higher overhead only to simulate the API functions used to initialize the RDMA device or allocate/deallocate RDMA buffers.

show abstract

Bridging the Virtualization Performance Gap for HPC Using SR-IOV for InfiniBand

Cited by 18 publications

References 14 publications

A Self-Adaptive Network for HPC Clouds: Architecture, Framework, and Implementation

A Self-Adaptive Network for HPC Clouds: Architecture, Framework, and Implementation

Enabling Diverse Software Stacks on Supercomputers Using High Performance Virtual Clusters

Lightweight and Generic RDMA Engine Para-Virtualization for the KVM Hypervisor

Contact Info

Product

Resources

About