Criticality Aware Tiered Cache Hierarchy: A Fundamental Relook at Multi-Level Cache Hierarchies

Nori, Anant; Gaur, Jayesh; Rai, Siddharth; Subramoney, Sreenivas; Hong, Wang

doi:10.1109/isca.2018.00019

Cited by 27 publications

(11 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Prefetch throttling mechanisms [32], [33], [44], [45], [53], [58], [74], [78], [80], [81], [88], [94] use dynamic information such as prefetch coverage/accuracy, cache pollution, and/or bandwidth utilization to monitor the aggressiveness of prefetches. These mechanisms can be applied to our approach to reduce prefetch-induced cache pollution.…”

Section: Related Workmentioning

confidence: 99%

Prodigy: Improving the Memory Latency of Data-Indirect Irregular Workloads Using Hardware-Software Co-Design

Talati

May

Behroozi

et al. 2021

2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA)

View full text Add to dashboard Cite

Irregular workloads are typically bottlenecked by the memory system. These workloads often use sparse data representations, e.g., compressed sparse row/column (CSR/CSC), to conserve space at the cost of complicated, irregular traversals. Such traversals access large volumes of data and offer little locality for caches and conventional prefetchers to exploit. This paper presents Prodigy, a low-cost hardware-software codesign solution for intelligent prefetching to improve the memory latency of several important irregular workloads. Prodigy targets irregular workloads including graph analytics, sparse linear algebra, and fluid mechanics that exhibit two specific types of datadependent memory access patterns. Prodigy adopts a "best of both worlds" approach by using static program information from software, and dynamic run-time information from hardware. The core of the system is the Data Indirection Graph (DIG)-a proposed compact representation used to express program semantics such as the layout and memory access patterns of key data structures. The DIG representation is agnostic to a particular data structure format and is demonstrated to work with several sparse formats including CSR and CSC. Program semantics are automatically captured with a compiler pass, encoded as a DIG, and inserted into the application binary. The DIG is then used to program a low-cost hardware prefetcher to fetch data according to an irregular algorithm's data structure traversal pattern. We equip the prefetcher with a flexible prefetching algorithm that maintains timeliness by dynamically adapting its prefetch distance to an application's execution pace.We evaluate the performance, energy consumption, and transistor cost of Prodigy using a variety of algorithms from the GAP, HPCG, and NAS benchmark suites. We compare the performance of Prodigy against a non-prefetching baseline as well as stateof-the-art prefetchers. We show that by using just 0.8KB of storage, Prodigy outperforms a non-prefetching baseline by 2.6× and saves energy by 1.6×, on average. Prodigy also outperforms modern data prefetchers by 1.5-2.3×.Index Terms-DRAM stalls, irregular workloads, graph processing, hardware-software co-design, programming model, programmer annotations, compiler, and hardware prefetching. Program the prefetcher Program the prefetcher Generate prefetch requests Generate prefetch requestsCompiler analysis Run application Software Hardware Instrumented application binary Application source code Add DIG representation Data Indirection Graph (DIG)

show abstract

Section: Related Workmentioning

confidence: 99%

Prodigy: Improving the Memory Latency of Data-Indirect Irregular Workloads Using Hardware-Software Co-Design

Talati

May

Behroozi

et al. 2021

2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA)

View full text Add to dashboard Cite

show abstract

“…This is actually not ideal, some instructions may be more damaging to the performance than others, as these instructions could cost more number of cycles. A lack of focus in such bottleneck-inducing issue forms the fundamental motivation for Criticality Aware Tiered Cache Hierarchy (CATCH), proposed by Nori et al [1] as a possible solution.…”

Section: A Solutionmentioning

confidence: 99%

“…For a processor with a reorder buffer size of 224, the total area required for this critical path identification on the hardware is about 3 KB. The area calculation is discussed in details in [1].…”

Section: A Solutionmentioning

confidence: 99%

See 1 more Smart Citation

A Survey of Novel Cache Hierarchy Designs for High Workloads

Rajput,

Dellarosa,

Satis

2021

Preprint

View full text Add to dashboard Cite

Traditional on-die, three-level cache hierarchy design is very commonly used but is also prone to latency, especially at the Level 2 (L2) cache. We discuss three distinct ways of improving this design in order to have better performance. Performance is especially important for systems with high workloads. The first method proposes to eliminate L2 altogether while proposing a new prefetching technique, the second method suggests increasing the size of L2, while the last method advocates the implementation of optical caches. After carefully contemplating results in performance gains and the advantages and disadvantages of each method, we found the last method to be the best of the three.

show abstract

“…Por questões de eficiência, a cache é normalmente organizada em múltiplas ca-madas em uma hierarquia, onde as caches mais próximas da memória principal são mais lentas e maiores, enquanto as mais próximas dos núcleos são menores e mais rápidas. A maioria dos processadores modernos de alto desempenho empregam três níveis de cache, com uma cache de nível 1 (L1) e nível 2 (L2), privadas para cada núcleo, e uma cache L3 compartilhada por múltiplos núcleos [Nori et al 2018].…”

Section: Poluição De Cache E Thrashingunclassified

Poluição de Cache e Thrashing em Aplicações Paralelas de Alto Desempenho

Krause

Moreira

Girelli

et al. 2019

Anais Do XX Simpósio Em Sistemas Computacionais De Alto Desempenho (SSCAD 2019)

View full text Add to dashboard Cite

Conforme os processadores evoluem, o desempenho dos sistemas computacionais se torna cada vez mais limitado pelo tempo de acesso à memória. Caches são empregadas a fim de contornar este problema, mas é necessária uma gerência inteligente dos dados que são armazenados nelas para impedir que problemas como poluição e thrashing degradem seu desempenho. Neste trabalho é apresentada uma análise da poluição de cache e thrashing em aplicações paralelas de alto desempenho. Os resultados mostram que caches com maior associatividade sofrem mais com estes problemas. Até 28% dos cache misses na L1 poderiam ser evitados com uma política de substituição de cache mais inteligente, chegando a até 62% na cache L2 e 98% na LLC. As processors evolve, the performance of computer systems becomes increasingly limited by the memory access time. Caches are employed in order to get around this problem, but an intelligent management of the data that is stored in them is necessary to prevent problems such as pollution and thrashing from degrading their performance. In this work, an analysis of cache and thrashing pollution in high performance parallel applications is presented. The results show that caches with greater associativity suffer more from these problems. Up to 28% of cache misses in the L1 cache could be avoided with a smarter replacement policy, up to 62% in the L2 cache and 98% in the LLC.

show abstract

Criticality Aware Tiered Cache Hierarchy: A Fundamental Relook at Multi-Level Cache Hierarchies

Cited by 27 publications

References 36 publications

Prodigy: Improving the Memory Latency of Data-Indirect Irregular Workloads Using Hardware-Software Co-Design

Prodigy: Improving the Memory Latency of Data-Indirect Irregular Workloads Using Hardware-Software Co-Design

A Survey of Novel Cache Hierarchy Designs for High Workloads

Poluição de Cache e Thrashing em Aplicações Paralelas de Alto Desempenho

Contact Info

Product

Resources

About