Reuse-distance-based miss-rate prediction on a per instruction basis

Abstract

Feedback-directed optimization has become an increasingly important tool in designing and building optimizing compilers. Recently, reuse-distance analysis has shown much promise in predicting the memory behavior of programs over a wide range of data sizes. Reuse-distance analysis predicts program locality by experimentally determining locality properties as a function of the data size of a program, allowing accurate locality analysis when the program's data size changes.Prior work has established the effectiveness of reuse distance analysis in predicting whole-program locality and miss rates. In this paper, we show that reuse distance can also effectively predict locality and miss rates on a per instruction basis. Rather than predict locality by analyzing reuse distances for memory addresses alone, we relate those addresses to particular static memory operations and predict the locality of each instruction.Our experiments show that using reuse distance without cache simulation to predict miss rates of instructions is superior to using cache simulations on a single representative data set to predict miss rates on various data sizes. In addition, our analysis allows us to identify the critical memory operations that are likely to produce a significant number of cache misses for a given data size. With this information, compilers can target cache optimization specifically to the instructions that can benefit from such optimizations most.

Pinpointing and Exploiting Opportunities for Enhancing Data Reuse

“…Previous work on computing cache miss predictions from memory reuse distance information has explored approaches that associate reuse distance data with either individual references [9], [13], groups of related references from the same loop [14], or an entire application [23]. Associating reuse distance data with a section of code, be it a reference, a loop or an entire application, is sufficient for computing the number of cache misses incurred by that piece of code.…”

“…These include investigating memory hierarchy management techniques [3], [16], characterizing data locality in program executions for individual program inputs [4], [7], and using memory reuse distance data from training runs to predict cache miss rate for other program inputs [9], [13], [23].…”

An Efficient Simulation Algorithm for Cache of Random Replacement Policy

“…In case of miss event, a random place from the cache is picked, regardless of whether it originally contains valid data or not. 2 The original content at the place will be evicted to allow storing the address A and its corresponding value just retrieved through backup mechanism. Two nearest references to the same cache line forms a reuse window, with references to other cache lines in between.…”

“…Hence a setassociative cache of size S and associativity M is equivalent to S M fully-associative cache operating in parallel, each of size M . 2 Even if the replacement algorithm takes care to not evict valid data when there are free slots, as the cache is soon filled up with valid data, there will be no difference in practice. 3 Sometimes referred to as pseudo random replacement policy due to difficulty, if not impossibility, of obtaining true randomness.…”

The Potential of Computation Regrouping for Improving Locality

“…A trace may not represent the program behavior on other inputs, and a trace may be too large to be analyzed. For many programs, earlier work has shown that the temporal locality follows a predictable pattern and the (cache miss) behavior of all program inputs can be predicted by examining medium-size training runs [14,15,26,34,44]. In this paper, we use a medium-size input for each program.…”