Bypass and insertion algorithms for exclusive last-level caches

Gaur, Jayesh; Chaudhuri, Mainak; Subramoney, Sreenivas

doi:10.1145/2000064.2000075

Cited by 62 publications

(31 citation statements)

References 23 publications

(13 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, a reuse distance can be a key measure to determine whether to bypass or not [4,5,6,7]. Some other works consider reuse counts for making a decision in cache bypassing [8,9,10,11]. In [12], cache bypass decision is performed by using support vector machine.…”

Section: Related Workmentioning

confidence: 99%

A DVFS-aware cache bypassing technique for multiple clock domain mobile SoCs

Kong

Lee

2017

IEICE Electron. Express

View full text Add to dashboard Cite

Multiple clock domains mobile SoCs typically adopt dynamic voltage and frequency scaling (DVFS) for flexible power/energy management. However, adoption of system-level cache under DVFS-enabled CPU may incur abnormal cache hierarchy (i.e., a delay reversal between the highlevel and low-level caches). It may lead to performance-and energyinefficiency due to slower data delivery and meaningless accesses to intermediate levels of caches. To resolve this problem, we propose a DVFS-aware cache bypassing technique. Our technique profiles latencies of the various levels of the caches. Based on the profiled information, our technique adaptively bypasses intermediate levels of caches in the case of abnormal cache hierarchy and applies power-gating to that cache memory for better energy efficiency. According to our evaluation, our technique reduces L2 and system-level cache energy consumption by up to 14.5% while improving performance by up to 0.13% compared to the baseline.

show abstract

Section: Related Workmentioning

confidence: 99%

A DVFS-aware cache bypassing technique for multiple clock domain mobile SoCs

Kong

Lee

2017

IEICE Electron. Express

View full text Add to dashboard Cite

show abstract

“…To make good use of the cache space, hot cache lines should be protected, and requests to conflicting addresses have to be bypassed when contention happens. Thus, existing CPU bypass policies [12], [11] can be applied to GPU to avoid pollution and thrashing access patterns.…”

Section: Cache Bypassing On Gpusmentioning

confidence: 99%

Adaptive Cache Management for Energy-Efficient GPU Computing

Chen

Chang

Rodrigues

et al. 2014

2014 47th Annual IEEE/ACM International Symposium on Microarchitecture

148

View full text Add to dashboard Cite

Abstract-With the SIMT execution model, GPUs can hide memory latency through massive multithreading for many applications that have regular memory access patterns. To support applications with irregular memory access patterns, cache hierarchies have been introduced to GPU architectures to capture temporal and spatial locality and mitigate the effect of irregular accesses. However, GPU caches exhibit poor efficiency due to the mismatch of the throughput-oriented execution model and its cache hierarchy design, which limits system performance and energy-efficiency.The massive amount of memory requests generated by GPUs cause cache contention and resource congestion. Existing CPU cache management policies that are designed for multicore systems, can be suboptimal when directly applied to GPU caches. We propose a specialized cache management policy for GPGPUs. The cache hierarchy is protected from contention by the bypass policy based on reuse distance. Contention and resource congestion are detected at runtime. To avoid oversaturating on-chip resources, the bypass policy is coordinated with warp throttling to dynamically control the active number of warps. We also propose a simple predictor to dynamically estimate the optimal number of active warps that can take full advantage of the cache space and on-chip resources. Experimental results show that cache efficiency is significantly improved and on-chip resources are better utilized for cachesensitive benchmarks. This results in a harmonic mean IPC improvement of 74% and 17% (maximum 661% and 44% IPC improvement), compared to the baseline GPU architecture and optimal static warp throttling, respectively.

show abstract

“…Gaur et al [7] proposed cache bypassing algorithms specific to exclusive cache hierarchies. The key contribution of this work is to tackle the problem of no reuse information being available for cache blocks (a cache hit de-allocates the cache block and the reuse information is lost).…”

Section: Related Workmentioning

confidence: 99%

“…A good cache replacement policy improves cache performance by selecting the least likely to be reused block as the victim and has been studied extensively [1] [3][5] [9][16] [17][20] [23]. A good cache allocation policy chooses to bypass a block to upper levels if it is predicted to be less useful than the blocks currently in the cache [7].…”

Section: Introductionmentioning

confidence: 99%

Adaptive Cache Bypassing for Inclusive Last Level Caches

Gupta

Gao

Zhou

2013

2013 IEEE 27th International Symposium on Parallel and Distributed Processing

View full text Add to dashboard Cite

Abstract-Cache hierarchy designs, including bypassing, replacement, and the inclusion property, have significant performance impact. Recent works on high performance caches have shown that cache bypassing is an effective technique to enhance the last level cache (LLC) performance. However, commonly used inclusive cache hierarchy cannot benefit from this technique because bypassing inherently breaks the inclusion property. This paper presents a solution to enabling cache bypassing for inclusive caches. We introduce a bypass buffer to an LLC. Bypassed cache lines skip the LLC while their tags are stored in this bypass buffer. When a tag is evicted from the bypass buffer, it invalidates the corresponding cache lines in upper level caches to ensure the inclusion property. Our key insight is that the lifetime of a bypassed line, assuming a well-designed bypassing algorithm, should be short in upper level caches and is most likely dead when its tag is evicted from the bypass buffer. Therefore, a small bypass buffer is sufficient to maintain the inclusion property and to reap most performance benefits of bypassing. Furthermore, the bypass buffer facilitates bypassing algorithms by providing the usage information of bypassed lines. We show that a top performing cache bypassing algorithm, which is originally designed for non-inclusive caches, performs comparably for inclusive caches equipped with our bypass buffer. The usage information collected from the bypass buffer also significantly reduces the cost of hardware implementation compared to the original design.

show abstract

Bypass and insertion algorithms for exclusive last-level caches

Cited by 62 publications

References 23 publications

A DVFS-aware cache bypassing technique for multiple clock domain mobile SoCs

A DVFS-aware cache bypassing technique for multiple clock domain mobile SoCs

Adaptive Cache Management for Energy-Efficient GPU Computing

Adaptive Cache Bypassing for Inclusive Last Level Caches

Contact Info

Product

Resources

About