Cellular Disco

Abstract

Despite the fact that large-scale shared-memory multiprocessors have been commercially available for several years, system software that fully utilizes all their features is still not available, mostly due to the complexity and cost of making the required changes to the operating system. A recently proposed approach, called Disco, substantially reduces this development cost by using a virtual machine monitor that leverages the existing operating system technology.In this paper we present a system called Cellular Disco that extends the Disco work to provide all the advantages of the hardware partitioning and scalable operating system approaches. We argue that Cellular Disco can achieve these benefits at only a small fraction of the development cost of modifying the operating system. Cellular Disco effectively turns a large-scale shared-memory multiprocessor into a virtual cluster that supports fault containment and heterogeneity, while avoiding operating system scalability bottlenecks. Yet at the same time, Cellular Disco preserves the benefits of a shared-memory multiprocessor by implementing dynamic, fine-grained resource sharing, and by allowing users to overcommit resources such as processors and memory. This hybrid approach requires a scalable resource manager that makes local decisions with limited information while still providing good global performance and fault containmenLIn this paper we describe our experience with a Cellular Disco prototype on a 32-processor SGI Origin 2000 system. We show that the execution time penalty for this approach is low, typically within 10% of the best available commercial operating system for most workloads, and that it can manage the CPU and memory resources of the machine significantly better than the hardware partitioning approach.

Sharing non‐cache‐coherent memory with bounded incoherence

“…Scale‐out systems distribute data across a cluster, 33 in some cases by relaxing consistency 34 . Some systems treat a single system as one that is distributed, 35‐37 and use message‐passing‐based coordination 9 . Message passing is traditionally used to implement distributed shared memory 28,29,31,38‐40 and provide partitioned global address space (PGAS) abstraction 41,42 .…”

Hardware and Software Support for Virtualization

“…Disco [44,81] was a research hypervisor designed for the Stanford FLASH multiprocessor [117] built using MIPS [85] processors. Disco is credited for its historical role in the resurgence of virtual machines [2].…”

“…Rather than attempting to modify existing operating systems to run on scalable shared-memory multiprocessors, Disco inserted an additional layer of software between the hardware and the operating system. In this design, the hypervisor handled the changes required to support scalable shared-memory computing, in particular resource management and transparent resource sharing (described in the original paper) and fault containment, described in Cellular Disco [81].…”

“…e combined effect of the three benefits yields a substantial performance improvement. For example, the Elvis system-which implements paravirtual block and net devices on sidecores-is up to 3x more performant than baseline paravirtualization and is oftentimes on par with SRIOV+ELI [83]. e sidecore approach has been successfully applied to various other tasks, including IOMMU virtualization [12], storage virtualization [34], GPU processing [159], and rack-scale computing [116].…”

Characteristic Analysis of Operating Systems for Large Scale Hierarchical NUMA System

“…OSes like Disco [9] and Cellular Disco [10] use virtual machines to run multiple commodity operating systems on a scalable multiprocessor and add a virtual machine monitor, a software layer between the hardware and multiple virtual machines that run independent operating systems, both systems build a partial single-system image using standard distributed systems protocols. Cellular Disco increases support for hardware fault containment and aggressive global resource management on the basis of Disco by combining the characteristic of Hive technology.…”