GPU Multisplit

Abstract

Multisplit is a broadly useful parallel primitive that permutes its input data into contiguous buckets or bins, where the function that categorizes an element into a bucket is provided by the programmer. Due to the lack of an e cient multisplit on GPUs, programmers often choose to implement multisplit with a sort. One way is to rst generate an auxiliary array of bucket IDs and then sort input data based on it. In case smaller indexed buckets possess smaller valued keys, another way for multisplit is to directly sort input data. Both methods are ine cient and require more work than necessary: the former requires more expensive data movements while the latter spends unnecessary e ort in sorting elements within each bucket. In this work, we provide a parallel model and multiple implementations for the multisplit problem. Our principal focus is multisplit for a small (up to 256) number of buckets. We use warp-synchronous programming models and emphasize warp-wide communications to avoid branch divergence and reduce memory usage. We also hierarchically reorder input elements to achieve better coalescing of global memory accesses. On a GeForce GTX 1080 GPU, we can reach a peak throughput of 18.93 Gkeys/s (or 11.68 Gpairs/s) for a key-only (or key-value) multisplit. Finally, we demonstrate how multisplit can be used as a building block for radix sort. In our multisplit-based sort implementation, we achieve comparable performance to the fastest GPU sort routines, sorting 32-bit keys (and key-value pairs) with a throughput of 3.0 G keys/s (and 2.1 Gpair/s).

Data-Parallel Hashing Techniques for GPU Architectures

“…The CUDA Data Parallel Primitives Library (CUDPP) [1] is a library of fundamental DPPs and algorithms written in Nvidia CUDA C [85] and designed for high-performance execution on CUDA-compatible GPUs. Each DPP and algorithm incorporated into the library is considered best-in-class and typically published in peer-reviewed literature (e.g., radix sort [6], [76], mergesort [27], [97], and cuckoo hashing [3], [4]). Thus, its data-parallel implementations are constantly updated to reflect the state-of-the-art.…”

WarpCore: A Library for fast Hash Tables on GPUs

“…The distributed mode assigns each key (and its associated values) to exactly one distinct GPU. This is done by first partitioning keys of an input batch according to their corresponding GPU ID by means of a device-sided multi-split [16] followed by scattering these segments to the GPUs where they belong. In case each participating GPU holds a separate input batch, we use an all-to-all communication primitive on NVLink connected systems [17] to simultaneously exchange segments between all GPUs.…”

GPU LSM: A Dynamic Dictionary Data Structure for the GPU

“…Our bulk strategy for cleanup is to 1) iteratively merge all occupied levels from the smallest to the largest (neglecting the LSB to preserve time ordering), 2) mark all unmarked stale elements (e.g., overwriting the LSBs), 3) compact all valid elements together (e.g., using a two-bucket multisplit [20] to collect all unmarked valid elements in stage 2), 4) add enough placebos, and 5) redistribute (already sorted) elements to different (new) levels. Since all levels are already sorted, merging them together iteratively is much faster than resorting all of them together.…”

“…The more elements to be removed the better. For example, with n = (2 6 − 1)b elements where b = 2 20 , cleanup operations when {10, 50}% of elements should be removed runs at {1870.2, 1828.2} M elements/s. A GPU LSM with roughly the same size (n = (2 7 − 1)b with b = 2 19 ) with {10, 50}% of elements removed results in {1842.5, 1794.3} M elements/s.…”