A fully associative software-managed cache design

Abstract

As DRAM access latencies approach a thousand instruction-execution times and onchip caches grow to multiple megabytes, it is not clear that conventional cache structures continue to be appropriate. Two key featuresfull associativity and software managementhave been used successfully in the virtual-memory domain to cope with disk access latencies. Future systems will need to employ similar techniques to deal with DRAM latencies. This paper presents a practical, fully associative, software-managed secondary cache system that provides performance competitive with or superior to traditional caches without OS or application involvement. We see this structure as the first step toward OS-and application-aware management of large on-chip caches.This paper has two primary contributions: a practical design for a fully associative memory structure, the indirect index cache (IIC), and a novel replacement algorithm, generational replacement, that is specifically designed to work with the IIC. We analyze the behavior of an IIC with generational replacement as a drop-in, transparent substitute for a conventional secondary cache. We achieve miss rate reductions from 8% to 85% relative to a 4-way associative LRU organization, matching or beating a (practically infeasible) fully associative true LRU cache. Incorporating these miss rates into a rudimentary timing model indicates that the IIC/generational replacement cache could be competitive with a conventional cache at today's DRAM latencies, and will outperform a conventional cache as these CPU-relative latencies grow.

An adaptive, non-uniform cache structure for wire-delay dominated on-chip caches

“…We use generational promotion to reduce the amount of copying required by a pure LRU mapping, while still approximating an LRU list mapped onto the physical topology of a bank set. Generational replacement was recently proposed by Hallnor et al for making replacement decisions in a software-managed UCA called the Indirect Index Cache [9]. In our scheme, when a hit occurs to a cache line, it is swapped with the line in the bank that is the next closest to the cache controller.…”

“…Kessler examined designs for multi-megabyte caches built with discrete components [17]. Hallnor and Reinhardt [9] studied a fully associative software-managed design for large on-chip L2 caches, but not did not consider non-uniform access times.…”

Scratchpad memory allocation for arrays in permutation graphs

“…Early approaches [5][6][7] are mostly static, that is, the data reside either in off-chip memory, or in SPM, and they are never transferred between off-chip memory and SPM during execution of the program. Static approaches are usually less efficient than dynamic ones [8].…”

“…Dynamic SPM allocation techniques are presented in [4,[8][9][10][11][12][13] and can be roughly divided into three classes, namely, software caching-based, ILP-based, and heuristic-based. Software caching techniques include the earliest proposed dynamic approaches [8,9]. The key idea behind software caching is to emulate the behavior of a hardware cache in software, but this incurs relatively high overhead [14].…”

“…Of the live ranges, 6 is equal to 7, let 7 ≺ 6, and the relationships among the other live ranges are obvious. By the construction method described in Definition 10 and if at any step there are multiple minimal elements in the set the one with the smallest number is chosen, then the ordering of the nodes inserted is [2,1,5,4,3,8,7,6,9]. According to the orientation method given in Theorem 7, the derived graph, as shown in Figure 6(b), is obviously a comparability graph.…”