Semantic locality and context-based prefetching using reinforcement learning

Peled, Leeor; Mannor, Shie; Weiser, Uri; Etsion, Yoav

doi:10.1145/2749469.2749473

Cited by 86 publications

(62 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, since we know that the stream represents accesses to structured data, constructed and used by an algorithm with some form of internal logic and recurrence, we have better chances of finding order in the underlying semantics that produced it. The semantic locality paradigm [45] aims to uncover relations between memory accesses that are consequential through the use of program context information. The premise behind that is that recurring semantic relations will likely involve similar control flows, specific data values, and spatiotemporal patterns.…”

Section: Learning Memory Access Patternsmentioning

confidence: 99%

“…In this article, we propose a neural network prefetcher that dynamically adapts to programs' memory access patterns. The prefetcher uses a variety of workload cues to learn these patterns, ranging from semantic program information [45] to traditional architectural information (e.g., PC, miss history, branch history). The prefetcher employs a small NN implemented using a systolic array.…”

Section: Introductionmentioning

confidence: 99%

“…• We evaluate the prefetcher using the gem5 simulator [8] over SPEC2006, Graph500, and a set of kernels. The evaluation shows the prefetcher outperforms a classic stride prefetcher by 21.3% on average and outperforms seven state-of-the-art prefetchers (SMS [56], GHB-PC/DC [43], VLDP [53], IMP [61], BOP [41], Domino [6], and context-RL [45]) by 7.5%-18% on average. Moreover, we show that the proposed prefetcher performs on average as well as having the best of the existing prefetchers for each benchmark (with outliers up to 37% over the best competitor), which suggests the prefetcher exploits all known opportunities, as well as previously undetected ones.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Neural Network Prefetcher for Arbitrary Memory Access Patterns

Peled

Weiser

Etsion

2019

ACM Trans. Archit. Code Optim.

Self Cite

View full text Add to dashboard Cite

Memory prefetchers are designed to identify and prefetch specific access patterns, including spatiotemporal locality (e.g., strides, streams), recurring patterns (e.g., varying strides, temporal correlation), and specific irregular patterns (e.g., pointer chasing, index dereferencing). However, existing prefetchers can only target premeditated patterns and relations they were designed to handle and are unable to capture access patterns in which they do not specialize. In this article, we propose a context-based neural network (NN) prefetcher that dynamically adapts to arbitrary memory access patterns. Leveraging recent advances in machine learning, the proposed NN prefetcher correlates program and machine contextual information with memory accesses patterns, using online-training to identify and dynamically adapt to unique access patterns exhibited by the code. By targeting semantic locality in this manner, the prefetcher can discern the useful context attributes and learn to predict previously undetected access patterns, even within noisy memory access streams. We further present an architectural implementation of our NN prefetcher, explore its power, energy, and area limitations, and propose several optimizations. We evaluate the neural network prefetcher over SPEC2006, Graph500, and several microbenchmarks and show that the prefetcher can deliver an average speedup of 21.3% for SPEC2006 (up to 2.3×) and up to 4.4× on kernels over a baseline of PC-based stride prefetcher and 30% for SPEC2006 over a baseline with no prefetching.

show abstract

Section: Learning Memory Access Patternsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Neural Network Prefetcher for Arbitrary Memory Access Patterns

Peled

Weiser

Etsion

2019

ACM Trans. Archit. Code Optim.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Peled et al propose a context-based cache prefetcher model [60], that detects semantic locality and issues prefetches based on it. Their approach is to implement a machine learning model in hardware that observes "contexts" of memory accesses during execution.…”

Section: Related Workmentioning

confidence: 99%

Data-driven spatial locality

Miucin

Fedorova

2018

Proceedings of the International Symposium on Memory Systems

View full text Add to dashboard Cite

Over the past decades, core speeds have been improving at a much higher rate than memory bandwidth. This has caused the performance bottlenecks in modern software to shift from computation to data transfers. Hardware caches were designed to mitigate this problem, based on the principles of temporal and spatial locality. However, with the increasingly irregular access patterns in software, locality is difficult to preserve. Researchers and practitioners devote a lot of time and effort to improving memory performance from the software side. This is done either by restructuring the code to make access patterns more regular, or by changing the layout of data in memory to better accommodate caching policies. Experts often use correlations between the access pattern of an algorithm and properties of the objects it operates on to devise new ways to lay data out in memory. Prior work has shown the memory layout design process to be largely manual and difficult enough to result in high level publications.Our contribution is a set of tools, techniques and algorithms for automatic extraction of correlations between data and access patterns of programs. In order to collect a sufficient level of details about memory accesses, we present a compiler-based access instrumentation framework called DI-NAMITE. Further, we introduce access graphs, a novel way of representing spatial locality properties of programs which are generated from memory access traces. We use access graphs as a basis for Hierarchical Memory Layouts -a novel algorithm for estimating performance improvements to be gained from better data layouts. Finally, we present our Data-Driven Spatial Locality techniques which use the information available from previous steps to automatically extract the correlations between data and access patterns commonly used by experts to inform better layout design.iii Lay SummaryOver the past decades, the disparity between processor and main memory speeds has grown significantly. Many important applications today suffer from poor memory performance. To improve these applications, experts manually tune the placement of data in memory. This process requires a deep understanding of the algorithm and the underlying hardware on which it runs. This work presents insights and analysis into how experts create performant memory layouts, and proposes new abstractions, algorithms and techniques to automate parts of the process. The proposed solutions have the potential of helping performance-minded engineers to improve data layout in their programs. iv

show abstract

“…In doing so, the data for that load will already be in one of the cache levels, therefore significantly shortening the latency of said load instruction. There exist a numerous amount of work on prefetching going from stride prefetching [33] to prefetching using a global history buffer [51] and even prefetchers that are trained using machine learning techniques [55].…”

Section: Hardware Prefetchingmentioning

confidence: 99%