GVPROF: A Value Profiler for GPU-Based Clusters

Zhou, Keren; Hao, Yueming; Mellor-Crummey, John; Meng, Xiaozhu; Liu, Xu

doi:10.1109/sc41405.2020.00093

Cited by 17 publications

(9 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Diogenes [50] overloads GPU memory copy APIs to analyze duplicate values copied to the GPU but it does not analyze patterns of value use by GPU kernels. The most related approach is GVProf [58], a value profiler for NVIDIA GPUs. While GVProf can identify value redundancies, it does not systematically categorize value patterns and cannot identify as many inefficiencies as ValueExpert can, as we describe in Section 7.…”

Section: Related Workmentioning

confidence: 99%

“…ValueExpert's offline analyzer adopts a bidirectional slicing algorithm [58] that derives a GPU memory instruction's access 1 Based on our experiments, we use a threshold of 33%. type based on instructions with known access types on its def-use chains.…”

Section: Value Pattern Recognitionmentioning

confidence: 99%

“…merged intervals will be copied to the CPU. Compared with existing instrumentation-based GPU tools [40,58] that perform data processing at instrumentation callbacks or on CPUs, our approach employs highly-parallel data processing using many GPU threads, little CPU-GPU transfer time, and concurrent execution of data processing and application kernels.…”

Section: Accelerating Coarse-grained Analysismentioning

confidence: 99%

See 2 more Smart Citations

ValueExpert: exploring value patterns in GPU-accelerated applications

Zhou

Hao

Mellor-Crummey

et al. 2022

Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

Self Cite

View full text Add to dashboard Cite

General-purpose GPUs have become common in modern computing systems to accelerate applications in many domains, including machine learning, high-performance computing, and autonomous driving. However, inefficiencies abound in GPU-accelerated applications, which prevent them from obtaining bare-metal performance. Performance tools play an important role in understanding performance inefficiencies in complex code bases. Many GPU performance tools pinpoint time-consuming code and provide high-level performance insights but overlook one important performance issueÐvalue-related inefficiencies, which exist in many GPU code bases. In this paper, we present ValueExpert, a novel tool to pinpoint value-related inefficiencies in GPU applications.ValueExpert monitors application execution to capture values produced and used by each load and store operation in GPU kernels, recognizes multiple value patterns, and provides intuitive optimization guidance. We address systemic challenges in collecting, maintaining, and analyzing voluminous performance data from many GPU threads to make ValueExpert applicable to complex applications. We evaluate ValueExpert on a wide range of welltuned benchmarks and applications, including PyTorch, Darknet, LAMMPS, Castro, and many others. ValueExpert is able to identify previously unknown performance issues and provide suggestions for nontrivial performance improvements with typically less than five lines of code changes. We verify our optimizations with application developers and upstream fixes to their repositories.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Value Pattern Recognitionmentioning

confidence: 99%

Section: Accelerating Coarse-grained Analysismentioning

confidence: 99%

See 1 more Smart Citation

ValueExpert: exploring value patterns in GPU-accelerated applications

Zhou

Hao

Mellor-Crummey

et al. 2022

Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…For example, Goroshov et al [27] use instrumentation to measure basic block latency and detect hot code regions. GVProf [28] instruments GPU memory instructions to profile value redundancies. In HPCToolkit, we use GT-Pin to measure instruction counts within GPU kernels.…”

Section: Related Workmentioning

confidence: 99%

“…Using binary instrumentation to collect instruction traces, we could reconstruct a GPU calling context tree on the CPU as an application executes. However, this method would have high overhead with frequent communication between CPUs and GPUs to copy traces [28].…”

Section: Gpu Calling Context Treementioning

confidence: 99%

Measurement and analysis of GPU-accelerated applications with HPCToolkit

et al. 2021

Self Cite

View full text Add to dashboard Cite

To address the challenge of performance analysis on the US DOE's forthcoming exascale supercomputers, Rice University has been extending its HPCToolkit performance tools to support measurement and analysis of GPU-accelerated applications. To help developers understand the performance of accelerated applications as a whole, HPCToolkit's measurement and analysis tools attribute metrics to calling contexts that span both CPUs and GPUs. To measure GPU-accelerated applications efficiently, HPCToolkit employs a novel wait-free data structure to coordinate monitoring and attribution of GPU performance. To help developers understand the performance of complex GPU code generated from high-level programming models, HPCToolkit constructs sophisticated approximations of call path profiles for GPU computations. To support fine-grained analysis and tuning, HPCToolkit uses PC sampling and instrumentation to measure and attribute GPU performance metrics to source lines, loops, and inlined code. To supplement fine-grained measurements, HPCToolkit can measure GPU kernel executions using hardware performance counters. To provide a view of how an execution evolves over time, HPCToolkit can collect, analyze, and visualize call path traces within and across nodes. Finally, on NVIDIA GPUs, HPCToolkit can derive and attribute a collection of useful performance metrics based on measurements using GPU PC samples. We illustrate HPCToolkit's new capabilities for analyzing GPU-accelerated applications with several codes developed as part of the Exascale Computing Project.

show abstract