Machine Learning for Fine-Grained Hardware Prefetcher Control

Hiebel, Jason; Brown, Laura E.; Wang, Zhenlin

doi:10.1145/3337821.3337854

Cited by 15 publications

(12 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ML methods have been gaining traction in place of heuristic methods for achieving superior prefetcher management performance [5], [22], [24], [32], [34]. ML algorithms can extract the non-intuitive interactions between the different prefetchers.…”

Section: B Ml-based Prefetcher Managersmentioning

confidence: 99%

See 1 more Smart Citation

Puppeteer: A Random Forest-based Manager for Hardware Prefetchers across the Memory Hierarchy

Eris¹,

Louis²,

Eris³

et al. 2022

Preprint

View full text Add to dashboard Cite

Over the years, processor throughput has steadily increased. However, the memory throughput has not increased at the same rate, which has led to the memory wall problem in turn increasing the gap between effective and theoretical peak processor performance. To cope with this, there has been an abundance of work in the area of data/instruction prefetcher designs. Broadly, prefetchers predict future data/instruction address accesses and proactively fetch data/instructions in the memory hierarchy with the goal of lowering data/instruction access latency. To this end, one or more prefetchers are deployed at each level of the memory hierarchy, but typically, each prefetcher gets designed in isolation without comprehensively accounting for other prefetchers in the system. As a result, individual prefetchers do not always complement each other, and that leads to lower average performance gains and/or many negative outliers. In this work, we propose Puppeteer, which is a hardware prefetcher manager that uses a suite of random forest regressors to determine at runtime which prefetcher should be ON at each level in the memory hierarchy, such that the prefetchers complement each other and we reduce the data/instruction access latency. Compared to a design with no prefetchers, using Puppeteer we improve IPC by 46.0% in 1 Core (1C), 25.8% in 4 Core (4C), and 11.9% in 8 Core (8C) processors on average across traces generated from SPEC2017, SPEC2006, and Cloud suites with ∼10KB overhead. Moreover, we also reduce the number of negative outliers by over 89%, and the performance loss of the worst-case negative outlier from 25% to only 5% compared to the state-of-the-art.

show abstract

Section: B Ml-based Prefetcher Managersmentioning

confidence: 99%

“…Other algorithms, such as neural networks, are too complex and their size increases prohibitively with the size of the dataset [5]. Moreover, some prior works focus on hardware adaptation only from the perspective of accuracy without worrying about the hardware implementation [22], [24], [32], [46].…”

Section: B Ml-based Prefetcher Managersmentioning

confidence: 99%

Puppeteer: A Random Forest-based Manager for Hardware Prefetchers across the Memory Hierarchy

Eris¹,

Louis²,

Eris³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The hardware prefetcher configurations provided by Intel 2 for post-Nehalem microarchitectures provide 4 bits (16 combinations) to control four prefetchers [8,18,26]:…”

Section: Prefetcher Configurationsmentioning

confidence: 99%

“…We tried to only use subsets of many features collected within a single NUMA+Prefetcher configuration, but did not achieve good results. 6 The exploration considers 21 NUMA configurations instead of 18 trained over remaining codes. If the training accuracy is consistently high across the different folds, then it indicates that the model is able to effectively generalize to unseen codes.…”

Section: Model Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

Modeling and optimizing NUMA effects and prefetching with machine learning

Barrera

Black-Schaffer

Casas

et al. 2020

Proceedings of the 34th ACM International Conference on Supercomputing

View full text Add to dashboard Cite

Both NUMA thread/data placement and hardware prefetcher configuration have significant impacts on HPC performance. Optimizing both together leads to a large and complex design space that has previously been impractical to explore at runtime. In this work we deliver the performance benefits of optimizing both NUMA thread/data placement and prefetcher configuration at runtime through careful modeling and online profiling. To address the large design space, we propose a prediction model that reduces the amount of input information needed and the complexity of the prediction required. We do so by selecting a subset of performance counters and application configurations that provide the richest profile information as inputs, and by limiting the output predictions to a subset of configurations that cover most of the performance. Our model is robust and can choose near-optimal NUMA+Prefetcher configurations for applications from only two profile runs. We further demonstrate how to profile online with low overhead, resulting in a technique that delivers an average of 1.68× performance improvement over a locality-optimized NUMA baseline with all prefetchers enabled. CCS CONCEPTS • Computer systems organization → Multicore architectures; • General and reference → Performance; Measurement; • Software and its engineering → Memory management; • Computing methodologies → Cluster analysis; Supervised learning; Crossvalidation; Model verification and validation; Model development and analysis.

show abstract

A prefetch control strategy based on improved hill-climbing method in asymmetric multi-core architecture

Fang

Kong

et al. 2023

J Supercomput

View full text Add to dashboard Cite

Cache prefetching is a traditional way to reduce memory access latency. In multi-core systems, aggressive prefetching may harm the system. In the past, prefetching throttling strategies usually set thresholds through certain factors. When the threshold is exceeded, prefetch throttling strategies will control the aggressive prefetcher. However, these strategies usually work well in homogeneous multi-core systems and do not work well in heterogeneous multi-core systems. This paper considers the performance difference between cores under the asymmetric multi-core architecture. Through the improved hill-climbing method, the aggressiveness of prefetching for different cores is controlled, and the IPC of the core is improved. Through experiments, it is found that compared with the previous strategy, the average performance of big core is improved by more than 3%, and the average performance of little cores is improved by more than 24%.

show abstract

Machine Learning for Fine-Grained Hardware Prefetcher Control

Cited by 15 publications

References 17 publications

Puppeteer: A Random Forest-based Manager for Hardware Prefetchers across the Memory Hierarchy

Puppeteer: A Random Forest-based Manager for Hardware Prefetchers across the Memory Hierarchy

Modeling and optimizing NUMA effects and prefetching with machine learning

A prefetch control strategy based on improved hill-climbing method in asymmetric multi-core architecture

Contact Info

Product

Resources

About