Reuse Distance-Based Cache Hint Selection

Abstract

Modern instruction sets extend their load/store-instructions with cache hints, as an additional means to bridge the processor-memory speed gap. Cache hints are used to specify the cache level at which the data is likely to be found, as well as the cache level where the data is stored after accessing it. In order to improve a program's cache behavior, the cache hint is selected based on the data locality of the instruction. We represent the data locality of an instruction by its reuse distance distribution. The reuse distance is the amount of data addressed between two accesses to the same memory location. The distribution allows to efficiently estimate the cache level where the data will be found, and to determine the level where the data should be stored to improve the hit rate. The Open64 EPIC-compiler was extended with cache hint selection. Execution on an Itanium multiprocessor shows speedups of up to 36% in numerical and 23% in non-numerical programs.

Automated locality optimization based on the reuse distance of string operations

“…[8] introduced some of the first key ideas about using the compiler to control cache bypass. [5] presents context based optimization. Unlike our approach, theirs records the reuse distance for each instruction and adds nontemporal hints if over 90% of the instances suggest nontemporal behavior (the actual decision depends on cache size and associativity).…”

Instruction based memory distance analysis and its application to optimization

“…Beyls and D'Hollander detect reuse distance patterns through profiling and generate hints for the Itanium processor [1]. It's unclear whether their profiling and experiments are on the same input or not, however, our work can be used to generate their hints.…”

“…either static analysis of regular array references [12,23] or profiling [1] to determine the locality of memory operations. Unfortunately, static analysis has limited applicability when index arrays or pointer operations are used in addressing and when determining locality across multiple loop nests.…”

An Efficient Simulation Algorithm for Cache of Random Replacement Policy

“…However, for an important category of cache, namely those of random replacement policy, as the replacement policy randomly determines one among multiple candidates for eviction, the naïve simulation only constitutes a Monte Carlo simulation, therefore can only give one out of many possible results. In particular, in a single round, naïve simulation cannot give the hit probability of each cache reference, which is of particular interest in program analysis [3] [4]. If we maintain n copies of possible cache states and simulate access sequences on these states simultaneously, then the time and space requirement of the simulation will be equal to running n copies of naïve simulation in parallel, with no gain in efficiency.…”

“…However, the probability distribution has M i=0 Z i size, and each transition matrix has O(M M i=0 Z i ) non-zero elements, where Z is the number of distinct elements in trace, rendering this simple approach infeasible in general. 4 Instead, we present an algorithm that implicitly calculates the probability distribution.…”