GDP: Using Dataflow Properties to Accurately Estimate Interference-Free Performance at Runtime

Abstract

Abstract-Multi-core memory systems commonly share resources between processors. Resource sharing improves utilization at the cost of increased inter-application interference which may lead to priority inversion, missed deadlines and unpredictable interactive performance. A key component to effectively manage multi-core resources is performance accounting which aims to accurately estimate interference-free application performance. Previously proposed accounting systems are either invasive or transparent. Invasive accounting systems can be accurate, but slow down latency-sensitive processes. Transparent accounting systems do not affect performance, but tend to provide less accurate performance estimates.We propose a novel class of performance accounting systems that achieve both performance-transparency and superior accuracy. We call the approach dataflow accounting, and the key idea is to track dynamic dataflow properties and use these to estimate interference-free performance. Our main contribution is Graph-based Dynamic Performance (GDP) accounting. GDP dynamically builds a dataflow graph of load requests and periods where the processor commits instructions. This graph concisely represents the relationship between memory loads and forward progress in program execution. More specifically, GDP estimates interference-free stall cycles by multiplying the critical path length of the dataflow graph with the estimated interference-free memory latency. GDP is very accurate with mean IPC estimation errors of 3.4% and 9.8% for our 4-and 8-core processors, respectively. When GDP is used in a cache partitioning policy, we observe average system throughput improvements of 11.9% and 20.8% compared to partitioning using the state-of-the-art Application Slowdown Model.

Selective Replication in Memory-Side GPU Caches

“…To predict the LLC misses for all possible replication degrees we devise a light-weight mechanism we call the Replication Degree Directory (RDD). The RDD is inspired by the Auxiliary Tag Directory (ATD) [55] which is an independent tag directory commonly used to predict per-application cache misses as a function of allocated ways in shared LLCs (see e.g., [27], [76]). Unlike an ATD, the RRD is (i) located within an MCrouter rather than an LLC slice, and (ii) predicts misses across replication degrees and not miss curves.…”

“…Herrero et al [24] use distributed cache partitioning to optimize cache use, and MorphCache [60] dynamically alters the cache topology to enable sharing multiple cache slices between cores. GDP [27] allocates LLC capacity to processes based on slowdown predictions, while Rolan et al [57] propose adaptive set-granular cooperative caching. These works are not directly applicable to GPUs as they exploit that different threads (processes) in multi-threaded (multiprogrammed) workloads have different memory requirements.…”

FirePerf

“…Abstract system-level simulators have long been used in the architecture and design automation communities for performance estimation and analysis [11,19,27,38,39,42,44,49,52]. In particular, [12] used system simulation to evaluate the interaction between the OS and a 10 Gbit/s Ethernet NIC.…”

HSM

“…Enforcing fairness/QoS requires understanding how interference affects the performance of co-running applications. More specifically, we need to predict the performance reduction (slowdown) during multitasking (shared mode) compared to an ideal configuration (private mode) where the application runs alone with exclusive access to all compute and memory system resources [10]. Using shared mode quantities (e.g., shared mode bandwidth utilization) as proxies for private mode quantities (e.g., private mode bandwidth utilization) is typically inaccurate since interference can change application resource consumption significantly.…”

“…Broadly speaking, slowdown prediction models can be classified as white-box [10,11,14] versus black-box [15,16]. White-box models are derived from fundamental architectural insights which enables them to, in theory, precisely capture key performance-related behavior.…”