Analytical modeling of set-associative cache behavior

Abstract

Cache behavior is complex and inherently unstable, yet is a critical factor a ecting program performance. A method of evaluating cache performance is required, both to give quantitative predictions of miss-ratio, and information to guide optimization of cache use. Traditional cache simulation gives accurate predictions of miss-ratio, but little to direct optimization. Also, the simulation time is usually far greater than the program execution time. Several analytical models have been developed, but concentrate mainly on direct-mapped caches, often for speci c types of algorithm, or to give qualitative predictions. In this work novel analytical models of cache phenomena are presented, applicable to numerical codes consisting mostly of array operations in looping constructs. Set-associative caches are considered, through an extensive hierarchy of cache reuse and interference e ects, including numerous forms of temporal and spatial locality. Models of each e ect are given, which, when combined, predict the overall miss-ratio. An advantage is that the models also indicate sources of cache interference. The accuracy of the models is validated through example program fragments. The predicted miss-ratios are compared with simulations, and shown typically to be within fteen percent. The evaluation time of the models is shown to be independent of the problem size, generally several orders of magnitude faster than simulation.

Cache Behavior Modeling of Codes with Data-Dependent Conditionals

“…Among those works we find [7], which is restricted to the modeling of direct-mapped caches and that lacks an automatic implementation. Later, [8] and [9] overcame some of these limitation. Cache Miss Equations (CMEs) are constructed in [8], which are lineal systems of Diophantine equations, where each solution corresponds to a potential miss cache.…”

Exact analysis of the cache behavior of nested loops

“…Harper et al [28] present an analytical model that focuses on set-associative caches. Their model approximates the cache missratio of a looping construct and allows imperfect loop nests to be considered.…”

Tuning Blocked Array Layouts to Exploit Memory Hierarchy in SMT Architectures

“…Although the results are accurate, the time needed to obtain them is typically many times greater than the total execution time of the program being simulated. To try to overcome such problems, analytical models of cache behaviour combined with heuristics have also been developed, to guide optimizing compilers [6], [16] and [23], or study the cache performance of particular types of algorithm, especially blocked ones [3], [7], [10], and [22]. Code optimizations, such as tile size selection, selected with the help of predicted miss ratios require a really accurate assessment of program's code behaviour.…”