Performance driven data cache prefetching in a dynamic software optimization system

Beyler, Jean Christophe; Clauss, Philippe

doi:10.1145/1274971.1275000

Cited by 14 publications

(5 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We hence include them to calculate the eventual priority of each cache line. As to the type of a cache line, due to the inaccuracy of the prefetcher, prefetched cache lines have a high likelihood of ending up as the dead ones [21] [22]. We give each cache line 1 bit to state whether it is prefetched or not.…”

Section: B Design Of Cbpmentioning

confidence: 99%

Reuse Distance-based Copy-backs of Clean Cache Lines to Lower-level Caches

Wang¹,

Wang²,

Ye³

2021

Preprint

View full text Add to dashboard Cite

Cache plays a critical role in reducing the performance gap between CPU and main memory. A modern multi-core CPU generally employs a multi-level hierarchy of caches, through which the most recently and frequently used data are maintained in each core's local private caches while all cores share the lastlevel cache (LLC). For inclusive caches, clean cache lines replaced in higher-level caches are not necessarily copied back to lower levels, as the inclusiveness implies their existences in lower levels. For exclusive and non-inclusive caches that are widely utilized by Intel, AMD, and ARM today, either indiscriminately copying back all or none of replaced clean cache lines to lower levels raises no violation to exclusiveness and non-inclusiveness definitions.We have conducted a quantitative study and found that, copying back all or none of clean cache lines to lower-level cache of exclusive caches entails suboptimal performance. The reason is that only a part of cache lines would be reused and others turn to be dead in a long run. This observation motivates us to selectively copy back some clean cache lines to LLC in an architecture of exclusive or non-inclusive caches. We revisit the concept of reuse distance of cache lines. In a nutshell, a clean cache line with a shorter reuse distance is copied back to lower-level cache as it is likely to be re-referenced in the near future, while cache lines with much longer reuse distances would be discarded or sent to memory if they are dirty. We have implemented and evaluated our proposal with non-volatile (STT-MRAM) LLC. Experimental results with gem5 and SPEC CPU 2017 benchmarks show that on average our proposal yields up to 12.8% higher throughput of IPC (instructions per cycle) than the least-recently-used (LRU) replacement policy with copying back all clean cache lines for STT-MRAM LLC.Index Terms-Mutli-level caches, clean cache lines, copy-back 1 We use 'copy back' for clean cache lines instead of 'write back' because 'write back' is conventionally used in literature to describe asynchronously updating dirty cache lines to lower levels of memory hierarchy. The latter is a contrast to the synchronous write-through strategy.

show abstract

Section: B Design Of Cbpmentioning

confidence: 99%

Reuse Distance-based Copy-backs of Clean Cache Lines to Lower-level Caches

Wang¹,

Wang²,

Ye³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…As a binary optimizer, ADORE [7] uses hardware counters to identify hotspots and phases and to apply memory prefetching optimizations. Similar goals are addressed in [5] where a dynamic system inserts prefetch instructions on-the-fly where it has been evaluated as effective by measuring the load latency. Both approaches focus on reducing the memory latency of memory instructions.…”

Section: Related Workmentioning

confidence: 99%

Runtime Vectorization Transformations of Binary Code

Hallou

Rohou

Clauss

2016

Int J Parallel Prog

Self Cite

View full text Add to dashboard Cite

International audienceIn many cases, applications are not optimized for the hardware on which they run. Several reasons contribute to this unsatisfying situation, such as legacy code, commercial code distributed in binary form, or deployment on compute farms. In fact, backward compatibility of ISA guarantees only the functionality, not the best exploitation of the hardware. In this work, we focus on maximizing the CPU efficiency for the SIMD extensions. The first contribution was originally published in the International Conference on Embedded Computer Systems: Architectures, Modeling and Simulation, SAMOS XV, Jul 2015, Agios Konstantinos, Greece. It is a binary-to-binary optimization framework where loops vectorized for an older version of the processor SIMD extension are automatically converted to a newer one. It is a lightweight mechanism that does not include a vectorizer, but instead leverages what a static vectorizer previously did. We show that many loops compiled for x86 SSE can be dynamically converted to the more recent and more powerful AVX; as well as, how correctness is maintained with regards to challenges such as data dependencies and reductions. We obtain speedups in line with those of a native compiler targeting AVX. The second contribution is the runtime vectorization of loops in binary codes that were not originally vectorized. For this purpose, we use open source frameworks that we have tuned and integrated to (1) dynamically lift the x86 binary into the Intermediate Representation form of the LLVM compiler, (2) abstract hot loops in the polyhedral model, (3) use the power of this mathematical framework to vectorize them, and (4) finally compile them back into executable form using the LLVM Just-In-Time compiler. In most cases, the obtained speedups are close to the number of elements that can be simultaneously processed by the SIMD unit. The re-vectorizer and auto-vectorizer are implemented inside a dynamic optimization platform; it is completely transparent to the user, does not require any rewriting of the binaries, and operates during program execution

show abstract

“…The final state is the prefetching of these delinquent loads. Another prefetch solution is dynamic prefetching as proposed by Beyler et al [3]. They studied a dynamic prefetch mechanism using the load latency variation to classify the loads.…”

Section: Related Workmentioning

confidence: 99%

“…This solution is based on hardware monitor of the Itanium processor. The two previous work [3] and [7] were done on Itanium architecture which is used for high performance computing. Our work is done on a light embedded VLIW processor which generally executes a single task; so, the dynamic prefetch mechanism is an inappropriate solution for our kind of architecture.…”

Section: Related Workmentioning

confidence: 99%