Minimal-overhead virtualization of a large scale supercomputer

Abstract

Virtualization has the potential to dramatically increase the usability and reliability of high performance computing (HPC) systems. However, this potential will remain unrealized unless overheads can be minimized. This is particularly challenging on large scale machines that run carefully crafted HPC OSes supporting tightlycoupled, parallel applications. In this paper, we show how careful use of hardware and VMM features enables the virtualization of a large-scale HPC system, specifically a Cray XT4 machine, with ≤5% overhead on key HPC applications, microbenchmarks, and guests at scales of up to 4096 nodes. We describe three techniques essential for achieving such low overhead: passthrough I/O, workload-sensitive selection of paging mechanisms, and carefully controlled preemption. These techniques are forms of symbiotic virtualization, an approach on which we elaborate.

Intelligent Management and Efficient Operation of Big Data

“…Consequently to obtain a good performance in the supporting infrastructure for processing big quantities of data, such as low latency and high throughput, some management enhancements are needed to operate more intelligently the available computing resources in each data center, such as virtual machines (Dai et al, 2013), memory (Zhou & Li, 2013), CPU scheduling (Bae et al, 2012), and cache (Koller et al, 2011), I/O (Ram et al, 2013). Other very important functional aspect to be aware in data centers is to enhance the network performance (Marx, 2013;Lange et al, 2011;Saleem, Hassan, & Asirvadam, 2011). To enhance this performance in a more flexible way, the network resources should become virtualized.…”

Dynamic adaptive virtual core mapping to improve power, energy, and performance in multi-socket multicores

“…This opportunity is leveraged in an automatic adaptive system that chooses between these two mappings. We implemented and evaluated in the context of the Palacios VMM [1,2,3] to do this. We demonstrate that the performance of SPEC and PARSEC benchmarks can be increased by as much as 66%, energy reduced by as much as 31%, and power reduced by as much as 17%, depending on the optimization objective.…”

A lightweight virtual machine monitor for Blue Gene/P

“…Like L4, Kitten also provides a native environment that can be used to develop customizations. The most notable differences are: that Palacios and Kitten are being developed for x86-based HPC systems rather than for a highly-specialized Supercomputer platform such as BG/P; that Palacios runs as a kernel module in Kitten's privileged domain, whereas our VMM runs completely decomposed and deprivileged as user-level process; and that their approach to device virtualization introduces a scheme relying on guest cooperation in order to achieving high-performance virtualization of HPC network devices [20], whereas our approach currently fully intercepts guest device accesses (although it could be adapted to support a similar scheme).…”