Page Placement Strategies for GPUs within Heterogeneous Memory Systems

Abstract

Systems from smartphones to supercomputers are increasingly heterogeneous, being composed of both CPUs and GPUs. To maximize cost and energy efficiency, these systems will increasingly use globally-addressable heterogeneous memory systems, making choices about memory page placement critical to performance. In this work we show that current page placement policies are not sufficient to maximize GPU performance in these heterogeneous memory systems. We propose two new page placement policies that improve GPU performance: one application agnostic and one using application profile information. Our application agnostic policy, bandwidth-aware (BW-AWARE) placement, maximizes GPU throughput by balancing page placement across the memories based on the aggregate memory bandwidth available in a system. Our simulation-based results show that BW-AWARE placement outperforms the existing Linux INTERLEAVE and LOCAL policies by 35% and 18% on average for GPU compute workloads. We build upon BW-AWARE placement by developing a compiler-based profiling mechanism that provides programmers with information about GPU application data structure access patterns. Combining this information with simple program-annotated hints about memory placement, our hint-based page placement approach performs within 90% of oracular page placement on average, largely mitigating the need for costly dynamic page tracking and migration.

Runtime Data Management on Non-Volatile Memory-based Heterogeneous Memory for Task-Parallel Programs

“…To reduce the data movement cost, we selectively place some data objects in DRAM at the beginning of the application, instead of placing all data objects in NVM. e existing work has demonstrated performance bene t of the initial data placement on GPU with HMS [1,25]. Our initial data placement technique on NVM-based HMS is consistent with those existing e orts.…”

“…BW d at a ob j = #dat a access × cacheline size #s ampl e s w i t h d at a acc e s s e s #s ampl e s × phase ex ecut ion t ime (1) e numerator of Equation 1 is the accessed data size. #data access in the numerator is the number of memory accesses to the data object in main memory.…”

“…ey introduce hardware modi cations to support massive data migration and performance optimization. Agarwal et al [1] introduce a bandwidth-aware data placement on GPU, driven by compiler extracted insights and explicit hints from programmers.…”

Work-In-Progress: Protecting Real-Time GPU Applications on Integrated CPU-GPU SoC Platforms

“…• Copy data from host memory to device (GPU) memory • Launch the function-called kernel-to be executed on the GPU • Wait until the kernel finishes • Copy the output from device memory to host memory In the real-time systems community, GPUs have been studied actively in recent years because of their potential benefits in accelerating demanding data-parallel real-time applications [5]. As observed in [6], GPU kernels typically demand high memory bandwidth to achieve high data parallelism and, if the memory bandwidth required by GPU kernels is not satisfied, it can result in significant performance reduction. For discrete GPUs, which have dedicated graphic memories, researchers have focused on addressing interference among the co-scheduled GPU tasks.…”

Hardware Translation Coherence for Virtualized Systems

“…Similarly, Micron's Hybrid Memory Cube [4,5] and byte-addressable persistent memories [6][7][8][9] are quickly gaining traction. Vendors are combining these high-performance memories with traditional high-capacity and low-cost DRAM, prompting research on heterogeneous memory architectures [2,[9][10][11][12][13][14][15].…”