Cache Calculus: Modeling Caches through Differential Equations

Much prior work has studied cache replacement, but a large gap remains between theory and practice. The design of many practical policies is guided by the optimal policy, Belady's MIN. However, MIN assumes perfect knowledge of the future that is unavailable in practice, and the obvious generalizations of MIN are suboptimal with imperfect information. What, then, is the right metric for practical cache replacement?We propose that practical policies should replace lines based on their economic value added (EVA), the difference of their expected hits from the average. Drawing on the theory of Markov decision processes, we discuss why this metric maximizes the cache's hit rate. We present an inexpensive implementation of EVA and evaluate it exhaustively. EVA outperforms several prior policies and saves area at iso-performance. These results show that formalizing cache replacement yields practical benefits. * This work was done while the author was at MIT. lacking a theoretical foundation, it is unclear if any are taking the right approach. Each policy performs well on particular programs, yet no policy dominates overall, suggesting that these policies are not making the best use of available information.This paper seeks to bridge theory and practice. We take a principled approach that builds on insights from recent empirical designs. First, we show that policies should replace candidates by their economic value added (EVA); i.e., how many more hits one expects from each candidate vs. the average candidate. Second, we design a practical implementation of this policy and show it outperforms existing policies. Contributions: Contributions:Contributions: Contributions: This paper contributes the following:• We discuss the two main tradeoffs in cache replacement: hit probability and cache space, and describe how EVA reconciles them in a single, intuitive metric. • We show that EVA maximizes the cache hit rate by drawing on Markov decision process (MDP) theory. • We present a practical implementation of EVA, which we have synthesized in a 65 nm commercial process. EVA adds 1% area on a 1 MB cache vs. SHiP. Our implementation is the first adaptive policy that does not require set sampling or auxiliary tag monitors. • We evaluate EVA against prior high-performance policies on SPEC CPU2006 and OMP2012 over many cache sizes. EVA reduces LLC misses over these policies at equal area, closing 57% of the gap from random replacement to MIN vs. 47% for SHiP [39], 41% for DRRIP [17], and 42% for PDP [14].Fewer misses translate into large area savings-EVA matches SHiP's performance with gmean 8% less total cache area. These contributions show that formalizing cache replacement yields practical benefits. EVA blends theory and practice to maximize upon available information and outperform many recent empirical policies. Beyond our particular design, we expect our analysis will prove useful in the design of future high-performance policies. Road map:Road map: Road map: Road map: Sec. II reviews prior approaches to cache replacement, and ...

show abstract

“…For interested readers, our technical report[6] presents a detailed cache replacement MDP using a reference model from our recent work[8,9].…”

mentioning

confidence: 99%

Maximizing Cache Performance Under Uncertainty

Beckmann

Sánchez

2017

2017 IEEE International Symposium on High Performance Computer Architecture (HPCA)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Indeed, the access patterns that cause large errors-highly correlated references-are precisely those that cache well in the private cache levels. In other work, we have shown that this assumption is surprisingly robust, yielding accurate predictions even on access patterns that violate it [6]. Sec.…”

Section: Model Assumptionsmentioning

confidence: 75%

“…then the coarsened hit equation is: The model is unchanged under coarsening for a deep reason. Our model can be relaxed into a system of differential equations through some simple transformations [6]. From this perspective, coarsening is essentially a numerical solution of the system using an adaptive step size [43].…”

Section: Increased Step Sizementioning

confidence: 99%

Modeling cache performance beyond LRU

Beckmann

Sánchez

2016

2016 IEEE International Symposium on High Performance Computer Architecture (HPCA)

Self Cite

View full text Add to dashboard Cite

Modern processors use high-performance cache replacement policies that outperform traditional alternatives like leastrecently used (LRU). Unfortunately, current cache models do not capture these high-performance policies as most use stack distances, which are inherently tied to LRU or its variants. Accurate predictions of cache performance enable many optimizations in multicore systems. For example, cache partitioning uses these predictions to divide capacity among applications in order to maximize performance, guarantee quality of service, or achieve other system objectives. Without an accurate model for high-performance replacement policies, these optimizations are unavailable to modern processors.We present a new probabilistic cache model designed for high-performance replacement policies. It uses absolute reuse distances instead of stack distances, and models replacement policies as abstract ranking functions. These innovations let us model arbitrary age-based replacement policies. Our model achieves median error of less than 1% across several high-performance policies on both synthetic and SPEC CPU2006 benchmarks. Finally, we present a case study showing how to use the model to improve shared cache performance.

show abstract

“…Unfortunately, they often require low-level system details which are not available till late in the design cycle. In contrast, analytical models-simpler ones in particularabstract away these low-level system details and provide key insights early in the design cycle that are useful for experts and non-experts alike [2,5,19,27,48,54].…”

Section: Introductionmentioning

confidence: 99%

LogCA

Altaf

Wood

2017

SIGARCH Comput. Archit. News

View full text Add to dashboard Cite

With the end of Dennard scaling, architects have increasingly turned to special-purpose hardware accelerators to improve the performance and energy efficiency for some applications. Unfortunately, accelerators don't always live up to their expectations and may underperform in some situations. Understanding the factors which effect the performance of an accelerator is crucial for both architects and programmers early in the design stage. Detailed models can be highly accurate, but often require low-level details which are not available until late in the design cycle. In contrast, simple analytical models can provide useful insights by abstracting away low-level system details. In this paper, we propose LogCA-a high-level performance model for hardware accelerators. LogCA helps both programmers and architects identify performance bounds and design bottlenecks early in the design cycle, and provide insight into which optimizations may alleviate these bottlenecks. We validate our model across a variety of kernels, ranging from sub-linear to super-linear complexities on both on-chip and off-chip accelerators. We also describe the utility of LogCA using two retrospective case studies. First, we discuss the evolution of interface design in SUN/Oracle's encryption accelerators. Second, we discuss the evolution of memory interface design in three different GPU architectures. In both cases, we show that the adopted design optimizations for these machines are similar to LogCA's suggested optimizations. We argue that architects and programmers can use insights from these retrospective studies for improving future designs. CCS CONCEPTS • Computing methodologies → Modeling methodologies; • Computer systems organization → Heterogeneous (hybrid) systems; • Hardware → Hardware accelerators;

show abstract

Cache Calculus: Modeling Caches through Differential Equations

Cited by 5 publications

References 11 publications

Maximizing Cache Performance Under Uncertainty

Maximizing Cache Performance Under Uncertainty

Modeling cache performance beyond LRU

LogCA

Contact Info

Product

Resources

About