Accelerometer

Sriraman, Akshitha; Dhanotia, Abhishek

doi:10.1145/3373376.3378450

Cited by 58 publications

(4 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Existing benchmarks that are generally used to design processors do not accurately represent the workloads at hyperscalars. As highlighted by several previous works [3,5,20,36,35], cloud workloads exhibit fundamentally different IPC, cache miss rates, and other metrics compared to standard benchmarks. These factors make the SPEC and other commercially available benchmarks a bad proxy for studying the performance of server processors in a datacenter.…”

Section: Code Memory Bw and Latency Challenges In Cloud Datacen-tersmentioning

confidence: 97%

“…They observe similar IPC, cache, and TLB MPKIs. Accelerometer [35] studies Meta's cloud workloads, finding most cycles spent on non-core application tasks, such as compression and serialization. Some earlier works [40,7] have also looked at µ-arch improvements for cloud workloads.…”

Section: Related Workmentioning

confidence: 99%

“…This paper is the first to study how cloud workloads interact with the memory system and to use these observations to propose shared µ-arch structures and memory bandwidth tiering. While previous works from Google [3,20] and Meta [36,35] have investigated the µ-arch behavior of hyperscale workloads, they have not performed detailed studies of the code behavior, memory bandwidth distribution, or hardware prefetcher efficiency. In summary, this paper makes the following contributions:…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Workload Behavior Driven Memory Subsystem Design for Hyperscale

Mahar¹,

Wang²,

Shu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Hyperscalars run services across a large fleet of servers, serving billions of users worldwide. However, these services exhibit different behaviors compared to commonly available benchmark suites, leading to server architectures that are suboptimal for cloud workloads. As datacenters emerge as the primary server processor market, optimizing server processors for cloud workloads by better understanding their behavior is an area of interest. To address this, we present MemProf, a memory profiler that profiles the three major reasons for stalls in cloud workloads: code-fetch, memory bandwidth, and memory latency. We use MemProf to understand the behavior of cloud workloads at Meta and propose and evaluate micro-architectural and memory system design improvements that help cloud workloads' performance.MemProf's code analysis shows that cloud workloads at Meta execute the same code across CPU cores. Using this, we propose shared micro-architectural structures-a shared L2 I-TLB and a shared L2 cache. Next, to help with memory bandwidth stalls, using workloads' memory bandwidth distribution, we find that only a few pages contribute to most of the system bandwidth. We use this finding to evaluate a new high-bandwidth, small-capacity memory tier and show that it performs 1.46× better than the current baseline configuration. Finally, we look into ways to improve memory latency for cloud workloads. Profiling using MemProf reveals that L2 hardware prefetchers, which are commonly used to reduce memory latency, have very low coverage and consume a significant amount of memory bandwidth. To help improve future hardware prefetcher performance, we built an efficient memory tracing tool to collect and validate production memory access traces. Our memory tracing tool adds significantly less overhead than DynamoRIO, enabling tracing production workloads.

show abstract

Section: Code Memory Bw and Latency Challenges In Cloud Datacen-tersmentioning

confidence: 97%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Workload Behavior Driven Memory Subsystem Design for Hyperscale

Mahar¹,

Wang²,

Shu³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Moreover, a large number of server-class CPUs are (and will be) used in datacenters to process High Performance Computing (HPC) workloads, and adding/maintaining additional accelerators just for DL workloads would increase complexity and cost [18]. Furthermore, offloading a modest-size task to an accelerator or a GPU might not be the best option if the offloading overhead is relatively substantial [48]. Finally, § Sana Damani and Eric Qin are now at NVIDIA and Meta, respectively.…”

Section: Introductionmentioning

confidence: 99%

VEGETA: Vertically-Integrated Extensions for Sparse/Dense GEMM Tile Acceleration on CPUs

Jeong¹,

Damani²,

Rajeshkumar³

et al. 2023

Preprint

View full text Add to dashboard Cite

Deep Learning (DL) acceleration support in CPUs has recently gained a lot of traction, with several companies (Arm, Intel, IBM) announcing products with specialized matrix engines accessible via GEMM instructions. CPUs are pervasive and need to handle diverse requirements across DL workloads running in edge/HPC/cloud platforms. Therefore, as DL workloads embrace sparsity to reduce the computations and memory size of models, it is also imperative for CPUs to add support for sparsity to avoid under-utilization of the dense matrix engine and inefficient usage of the caches and registers. This work presents VEGETA, a set of ISA and microarchitecture extensions over dense matrix engines to support flexible structured sparsity for CPUs, enabling programmable support for diverse DL models with varying degrees of sparsity. Compared to the state-of-theart (SOTA) dense matrix engine in CPUs, a VEGETA engine provides 1.09×, 2.20×, 3.74×, and 3.28× speed-ups when running 4:4 (dense), 2:4, 1:4, and unstructured (95%) sparse DNN layers.

show abstract