Reuse-based online models for caches

Abstract

We develop a reuse distance/stack distance based analytical modeling framework for efficient, online prediction of cache performance for a range of cache configurations and replacement policies LRU, PLRU, RANDOM, NMRU. Our framework unifies existing cache miss rate prediction techniques such as Smith's associativity model, Poisson variants, and hardware way-counter based schemes. We also show how to adapt LRU way-counters to work when the number of sets in the cache changes. As an example application, we demonstrate how results from our models can be used to select, based on workload access characteristics, last-level cache configurations that aim to minimize energy-delay product.

Cache Calculus: Modeling Caches through Differential Equations

“…Adaptive methods could be used to improve efficiency, while providing rigorous error bounds unlike prior models. Otherwise, this application of cache calculus is similar to prior reuse-distance-based models [3,4,8].…”

“…Prior models [1,3,4,5,6,8] aim to provide an efficient alternative to simulation, to, for example, accelerate design space exploration [1,5,8] or perform dynamic cache partitioning [3]. By contrast, our main goal is to augment simulation through simpler, closed-form models.…”

“…But despite this accuracy, the discrete model equations do not readily yield insight into the relationship between the cache size S or access pattern D. Moreover, it requires iteration that is orders of magnitude more expensive than cache calculus's models. These problems afflict all prior, general-purpose models [1,4,6,8], and other models [5,7] are limited to specific access patterns.…”

Modeling cache performance beyond LRU

“…Second, since modern LLCs use hashing and achieve high effective associativity, replacement candidates form a representative sample of cached lines [5,41,44]. Details of array organization, which have been the focus of prior studies [1,15,36,47,55], are relatively unimportant in modern LLCs. Our model leverages this insight by modeling replacement as a probabilistic process affecting individual lines, not sets.…”

“…Existing cache models [1,15,36,47,55] tend to focus on traditional, set-associative caches using simple replacement policies like least-recently used (LRU), pseudo-LRU, or random replacement. But modern processors do not use LRU (or pseudo-LRU) for the LLC. Modern LLCs instead employ high-performance replacement policies that greatly improve cache performance over traditional policies like LRU (Sec.…”