Are dynamic memory managers on GPUs slow?

Winter, Martin; Parger, Mathias; Mlakar, Daniel; Steinberger, Markus

doi:10.1145/3437801.3441612

Cited by 18 publications

(13 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, there is also a strong movement of developing system software on GPUs, such as database management systems. [27,37,[43][44][45] Dynamic memory allocation on GPUs was first introduced about ten years ago by NVIDIA and many other solutions have been proposed since then [42]. Many GPU-based applications benefit from dynamic memory allocation such as graph analytics [9,41], data analytics [5,35], and databases [4,19].…”

Section: Introductionmentioning

confidence: 99%

“…There are unique challenges in developing system software on massively parallel hardware, mostly imposed by the need to support a large number of parallel threads efficiently and the architectural complexity of the GPU hardware. Dynamic memory allocators in particular face challenges such as thread contention and synchronization overhead, and multiple studies [42] have proposed solutions to address these challenges. Similar to traditional memory allocators, such solutions utilize a shared data structure to keep track of available memory units [42].…”

Section: Introductionmentioning

confidence: 99%

“…Dynamic memory allocators in particular face challenges such as thread contention and synchronization overhead, and multiple studies [42] have proposed solutions to address these challenges. Similar to traditional memory allocators, such solutions utilize a shared data structure to keep track of available memory units [42]. For example, the state-of-theart solution, Ouroboros, uses a combination of linked-lists, arrays, and queues to reduce thread contention and memory fragmentation and was shown to outperform previous solutions in a recent comparative study [42].…”

Section: Introductionmentioning

confidence: 99%

“…Similar to traditional memory allocators, such solutions utilize a shared data structure to keep track of available memory units [42]. For example, the state-of-theart solution, Ouroboros, uses a combination of linked-lists, arrays, and queues to reduce thread contention and memory fragmentation and was shown to outperform previous solutions in a recent comparative study [42]. Nevertheless, we show in Section 2 that thread contention, synchronization overhead, and memory overhead are still problematic with such solutions.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Dynamic memory management in massively parallel systems

Pham

Yuan

et al. 2022

Proceedings of the 36th ACM International Conference on Supercomputing

View full text Add to dashboard Cite

Due to the high level of parallelism, there are unique challenges in developing system software on massively parallel hardware such as GPUs. One such challenge is designing a dynamic memory allocator whose task is to allocate memory chunks to requesting threads at runtime. State-of-the-art GPU memory allocators maintain a global data structure holding metadata to facilitate allocation/deallocation. However, the centralized data structure can easily become a bottleneck in a massively parallel system. In this paper, we present a novel approach for designing dynamic memory allocation without a centralized data structure. The core idea is to let threads follow a random search procedure to locate free pages. Then we further extend to more advanced designs and algorithms that can achieve an order of magnitude improvement over the basic idea. We present mathematical proofs to demonstrate that (1) the basic random search design achieves asymptotically lower latency than the traditional queue-based design and (2) the advanced designs achieve significant improvement over the basic idea. Extensive experiments show consistency to our mathematical models and demonstrate that our solutions can achieve up to two orders of magnitude improvement in latency over the best-known existing solutions.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Dynamic memory management in massively parallel systems

Pham

Yuan

et al. 2022

Proceedings of the 36th ACM International Conference on Supercomputing

View full text Add to dashboard Cite

show abstract

“…This necessitates a more sophisticated task-tohardware mapping for GPU than that for CPU. • Input Awareness: Dynamic memory allocation is expensive in GPU [112] and we can avoid it if we can estimate the worst-case memory usage. This can be done by using some meta information such as the maximum degree of the input graph.…”

Section: Introductionmentioning

confidence: 99%

Efficient and Scalable Graph Pattern Mining on GPUs

Chen¹,

Arvind²

2021

Preprint

View full text Add to dashboard Cite

We describe G 2 Miner, the first Graph Pattern Mining (GPM) framework that runs on multiple GPUs. G 2 Miner uses pattern-aware, input-aware and architecture-aware search strategies to achieve high efficiency on GPUs. To simplify programming, it provides a code generator that automatically generates pattern-aware CUDA code. G 2 Miner flexibly supports both breadth-first search (BFS) and depth-first search (DFS) to maximize memory utilization and generate sufficient parallelism for GPUs. For the scalability of G 2 Miner, we propose a customized scheduling policy to balance work among multiple GPUs. Experiments on a V100 GPU show that G 2 Miner is 5.4× and 7.2× faster than the two state-ofthe-art single-GPU systems, Pangolin and PBE, respectively. In the multi-GPU setting, G 2 Miner achieves linear speedups from 1 to 8 GPUs, for various patterns and data graphs. We also show that G 2 Miner on a V100 GPU is 48.3× and 15.2× faster than the state-of-the-art CPU-based systems, Peregrine and GraphZero, on a 56-core CPU machine.

show abstract