Adaptive Caches: Effective Shaping of Cache Behavior to Workloads

Subramanian, Ranjith; Smaragdakis, Yannis; Loh, Gabriel H.

doi:10.1109/micro.2006.7

Cited by 70 publications

(38 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The latter employs additional components including a skewed bloom filter in conjunction with a pipelined priority heap to identify and retain Copyright c 2013 The Institute of Electronics, Information and Communication Engineers the blocks that most frequently missed in the conventional part of the cache in the recent past. More adaptive cache management approaches can be found in [6], [7]. Although they successfully combine LRU and LFU replacement policies together via additional data structures, they significantly increase hardware overhead and complexity.…”

Section: Existing Solutionsmentioning

confidence: 99%

Adaptive Insertion and Promotion Policies Based on Least Recently Used Replacement

Jin

Yu³

et al. 2013

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYTo improve Last-Level Cache (LLC) management, numerous approaches have been proposed requiring additional hardware budget and increased overhead. A number of these approaches even change the organization of the existing cache design. In this letter, we propose Adaptive Insertion and Promotion (AIP) policies based on Least Recently Used (LRU) replacement. AIP dynamically inserts a missed line in the middle of the cache list and promotes a reused line several steps left, realizing the combination of LRU and LFU policies deliberately under a single unified scheme. As a result, it benefits workloads with high locality as well as with many frequently reused lines. Most importantly, AIP requires no additional hardware other than a typical LRU list, thus it can be easily integrated into the existing hardware with minimal changes. Other issues around LLC such as scans, thrashing and dead lines are all explored in our study. Experimental results on the gem5 simulator with SPEC CUP2006 benchmarks indicate that AIP outperforms LRU replacement policy by an average of 5.8% on the misses per thousand instructions metric.

show abstract

Section: Existing Solutionsmentioning

confidence: 99%

Adaptive Insertion and Promotion Policies Based on Least Recently Used Replacement

Jin

Yu³

et al. 2013

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…Workloads with little sharing can benefit from dynamic migration [15], but aggressive sharing requires careful tradeoff between replication and migration. Furthermore, LRU-based cache replacement performs well for workloads with good temporal locality, while frequency-based policies (e.g., LFU) are more suitable for workloads with poor locality [142].…”

Section: Diverse Workload Characteristicsmentioning

confidence: 99%

“…LRU is arguably the most widely used policy because its implementation is simpler than LFU and it can quickly adapt to working set changes. To provide good caching performance for a wide range of workloads, many software policies (e.g., [77,106]) and hardware designs (e.g., [42,142]) are proposed to combine the benefits of both LRU and LFU. CC achieves the same goal, but instead by using cache partitioning to isolate workloads with weak locality from those with good locality and by integrating LRU replacement with cache partitioning.…”

Section: Cache Replacement and Placementmentioning

confidence: 99%

Cooperative Caching for Chip Multiprocessors

Chang¹,

Herrero²,

Canal³

et al. 2011

Cooperative Networking

View full text Add to dashboard Cite

Chip multiprocessor (CMP) systems have made the on-chip caches a critical resource shared among co-scheduled threads. Limited off-chip bandwidth, increasing on-chip wire delay, destructive inter-thread interference, and diverse workload characteristics pose key design challenges. To address these challenge, we propose CMP cooperative caching (CC), a unified framework to efficiently organize and manage on-chip cache resources. By forming a globally managed, shared cache using cooperative private caches. CC can effectively support two important caching applications: (1) reduction of average memory access latency and (2) isolation of destructive inter-thread interference.CC reduces the average memory access latency by balancing between cache latency and capacity optimizations. Based private caches, CC naturally exploits their access latency benefits. To improve the effective cache capacity, CC forms a "shared" cache using replication control and LRU-based global replacement policies. Via cooperation throttling, CC provides a spectrum of caching behaviors between the two extremes of private and shared caches, thus enabling dynamic adaptation to suit workload requirements. We show that CC can achieve a robust performance advantage over private and shared cache schemes across different processor, cache and memory configurations, and a wide selection of multithreaded and multiprogrammed workloads.To isolate inter-thread caching interference, we add a time-sharing aspect on top of spatial cache partitioning. Our approach uses Multiple Time-sharing Partitions (MTP) to simultaneously improve throughput and fairness while maintaining QoS over the longer term. Each MTP partition unfairly improves at least one thread's throughput, and partitions favoring different threads are scheduled in a cooperative, timesharing manner to either maintain fairness and QoS, or implement priority. We also integrate MTP with CC's LRU-based capacity sharing policy to combine their benefits. The integrated scheme-Cooperative Caching Partitioning (CCP)-divides the total execution epochs into those controlled by either MTP or the ii baseline CC policy, respectively, according to the fraction of threads that can benefit from each of them. Our simulation results show that for a wide range of multiprogrammed workloads, CCP can improve throughput, fairness and QoS for workloads suffering from destructive interference, while achieving the performance benefit of the baseline CC policy for other workloads.iii

show abstract

“…The recently proposed adaptive replacement policy [12] provides support for multiple replacement policies and dynamically picks the best performing policy on a per set basis. They conclude that a combination of LRU and LFU performs fairly well for an L2 cache.…”

Section: Related Workmentioning

confidence: 99%

“…The placement mechanism and associativity are simultaneously tuned in Recent proposals like the V-way cache [10] and adaptive cache compression [1] simultaneously tune the placement mechanism and associativity by decoupling the tag portion of the cache from the data portion. On the other hand, proposals like the dynamic insertion policy [11] and adaptive replacement [12] are examples of schemes that modify the replacement mechanism. While placement and associativity are critical path functionalities and have a direct impact on access time, replacement decisions are made off the critical path.…”

Section: Introductionmentioning

confidence: 99%

Emulating Optimal Replacement with a Shepherd Cache

Rajan

Govindarajan

2007

40th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO 2007)

View full text Add to dashboard Cite

The inherent temporal locality in memory accesses is filtered out by the L1 cache. As a consequence, an L2 cache with LRU replacement incurs significantly higher misses than the optimal replacement policy (OPT). We propose to narrow this gap through a novel replacement strategy that mimics the replacement decisions of OPT. The L2 cache is logically divided into two components, a Shepherd Cache (SC) with a simple FIFO replacement and a Main Cache (MC) with an emulation of optimal replacement. The SC plays the dual role of caching lines and guiding the replacement decisions in MC. Our proposed organization can cover 40% of the gap between OPT and LRU for a 2MB cache resulting in 7% overall speedup. Comparison with the dynamic insertion policy, a victim buffer, a V-Way cache and an LRU based fully associative cache demonstrates that our scheme performs better than all these strategies.

show abstract

Adaptive Caches: Effective Shaping of Cache Behavior to Workloads

Cited by 70 publications

References 21 publications

Adaptive Insertion and Promotion Policies Based on Least Recently Used Replacement

Adaptive Insertion and Promotion Policies Based on Least Recently Used Replacement

Cooperative Caching for Chip Multiprocessors

Emulating Optimal Replacement with a Shepherd Cache

Contact Info

Product

Resources

About