Improving cache performance by selective cache bypass

Abstract

In traditional cache-based computers, all memory references are made through cache. However, a significant number of items which are referenced in a program are referenced so infrequently that other cache traffic is certain t o "bump" these items from cache before they are referenced again. In such cases, not only is there no benefit in placing the item in cache, but there is the additional overhead of "bumping" some other item out of cache to make room for this useless cache entry. Where a cache line is larger than a processor word, there is a n additional penalty in loading the entire line from memory into cache, whereas the reference could have been satisfied with a single word fetch. Simulations have shown t h a t these effects typically degrade cache-based system performance (average reference time) by 10% to 30%.This performance loss is due to cache pollution; by simply forcing "polluting" references to directly reference main memory -bypassing the cache -much of this performance can be regained. The technique proposed in this paper involves the use of new hardware, called a Bypass-Cache, which, under program control, will determine whether each reference should be through the cache or bypassing the cache and referencing main memory directly. Several inexpensive heuristics for the compiler to determine how to make each reference are given.

Load Miss Prediction - Exploiting Power Performance Trade-offs

“…The importance of reducing memory access latencies is reflected in a rich set of earlier results towards faster loads [26]- [29].…”

“…Software cache bypassing schemes were discussed by Chi [29]. Energy savings for scientific applications were considered by Choi, et al [4] and Freeh, et al [30].…”

Probabilistic Directed Writebacks for Exclusive Caches

“…Chi and Dietz [6] present an early work on selective cache bypassing; they use compiler support. Etsion et al [8] point out that if a resident block in a cache is chosen at random, it is unlikely to be a highly-referenced block, but if an access is chosen at random, it is likely to be to a highly-referenced block.…”

Optimal bypass monitor for high performance last-level caches

“…Based on how to predict distant reuse blocks, these studies can be classified into PC based [5,8,33] and address based [13,15,30,31]. LRF [37] combines PC based and address based methods to improve performance.…”