Architectural support for operating system-driven CMP cache management

Abstract

The role of the operating system (OS) in managing shared resources such as CPU time, memory, peripherals, and even energy is well motivated and understood [23]. Unfortunately, one key resource-lower-level shared cache in chip multi-processors-is commonly managed purely in hardware by rudimentary replacement policies such as least-recentlyused (LRU). The rigid nature of the hardware cache management policy poses a serious problem since there is no single best cache management policy across all sharing scenarios. For example, the cache management policy for a scenario where applications from a single organization are running under "best effort" performance expectation is likely to be different from the policy for a scenario where applications from competing business entities (say, at a third party data center) are running under a minimum service level expectation. When it comes to managing shared caches, there is an inherent tension between flexibility and performance. On one hand, managing the shared cache in the OS offers immense policy flexibility since it may be implemented in software. Unfortunately, it is prohibitively expensive in terms of performance for the OS to be involved in managing temporally fine-grain events such as cache allocation. On the other hand, sophisticated hardware-only cache management techniques to achieve fair sharing or throughput maximization have been proposed. But they offer no policy flexibility. This paper addresses this problem by designing architectural support for OS to efficiently manage shared caches with a wide variety of policies. Our scheme consists of a hardware cache quota management mechanism, an OS interface and a set of OS level quota orchestration policies. The hardware mechanism guarantees that OS-specified quotas are enforced in shared caches, thus eliminating the need for (and the performance penalty of) temporally fine-grained OS intervention. The OS retains policy flexibility since it can tune the quotas during regularly scheduled OS interventions. We demonstrate that our scheme can support a wide range of policies including policies that provide (a) passive per-

Effective Management of DRAM Bandwidth in Multicore Processors

“…Our second innovation is a feedback-based adaptive bandwidth sharing policy in which we periodically tune the bandwidth assigned to the sharers in order to achieve specified DRAM latencies. Adaptive bandwidth management does not add to the complexity of the hardware because it can be done entirely in software at the operating system (OS) or the hypervisor by using interfaces and mechanisms similar to those proposed for shared cache management [29]. While that interface supports various features such as thread migration and thread grouping (wherein a group of threads acts as a single resource principal), our study assumes that each thread running on a processor is a unique resource principal with its own allocated bandwidth share.…”

Cache Sharing Management for Performance Fairness in Chip Multiprocessors

“…Figure 7 shows the throughput of selected benchmark pairs with uncontrolled Pseudo LRU and the two cache partition schemes. We use fair speedup defined in Equation 14 to measure the throughput of concurrent running benchmark pairs. Note that taller bar in the figure means higher throughput.…”

“…[4] proposed a framework to provide QoS for resources including shared caches. [14] designed architectural support for OS to manage shared caches.…”

A Quantitative Study of Memory System Interference in Chip Multiprocessor Architectures

“…However, most of these studies have focused on a single component of the entire system. For example, techniques have been proposed to reduce cache capacity interference [1,[3][4][5][6][7], cache bandwidth interference [13] and memory bus transfer interference [2,8,9]. Unfortunately, a technique that reduces interference in one component is not adequate to provide interference control for the complete memory system.…”

“…Since these effects are clearly undesirable, there is a need for architectural techniques that provide predictable performance and improve fairness. Previously, cache capacity interference has received a great deal of attention [1,[3][4][5][6][7] while only a few researchers have proposed techniques that reduce memory bus interference [2,8,9]. Furthermore, there has been little interest in the details of designing a complete, thread-aware memory system [10][11][12].…”