Theoretically-Efficient and Practical Parallel In-Place Radix Sorting

Abstract

Radix sort stands out for having a better worse case theoretical bound than any comparison-based sort, for fixed length integers. Despite the fact that radix sort can be implemented either in-place or in parallel, there exists no parallel in-place implementation for radix sort that guarantees a sub-linear worst case span. The challenge arises due to read-write races when reading from and writing to the same array. In this thesis, I introduce Regions sort and use it to implement a parallel work-efficient in-place radix sorting algorithm. To sort integers from a range , and a parameter , the algorithm requires only (log log) auxiliary memory, (log) work and ((/ + log) log) span. Regions sort uses a divide-and-conquer paradigm. It first divides the array into sub-arrays, each of which is sorted independently in parallel. This decreases the irregularity in the input and reorders it into a set of regions, each of which has similar properties. Next, it builds a data structure, called the regions graph to represent the needed data movements to completely sort the array. Finally, Regions sort iteratively plans swaps that satisfies the required data movements. The algorithm then has to recurse on records with so-far equal keys to break ties. I compare two variants of Regions sort with the state-of-the-art sorting integer sort and comparison sort algorithms. Experiments show that the best variant of Regions sort usually outperforms other sorting algorithms. In addition, I perform different experiments showing that Regions sort maintains its superiority regardless input distribution and range. More importantly, the single-threaded implementation of Regions sort outperforms the highly optimized Ska Sort implementation for serial in-place radix sort.

The Case for a Learned Sorting Algorithm

“…Note that, in order to use Radix Sort with IEEE-754 floating point numbers, it is first necessary to shift and mask the bit representation. While Radix Sort is highly sensitive to the key length, which dictates the number of passes, it is nevertheless a very efficient sorting algorithm for numerical types, that is very well-suited for multi-core procedures [6,22,40], and SIMD vectorization [50].…”

“…As baselines, we compare against cache-optimized and highly tuned C++ implementations of Radix Sort [51], Timsort [18], Introsort (std::sort), Histogram Sort [4], and IS 4 o [49] (one of the most optimized sorting algorithms we were able to find, which was also recently used in other studies [40] as a comparison point). Note that we use a recursive, equidepth version of Histogram sort that adapts to the input's skew as to avoid severe performance penalties.…”

“…Our In fact, our learned sorting algorithm provides the best performance even when we include the model training time as a part of the overall sorting time. For example, our experiments show that Learned Sort yields an average of 3.38× performance improvement over C++ STL sort (std::sort)[16], 5.54× improvement over Timsort (Python's default sorting algorithm [45]), 1.49× over Radix sort [51], and 1.31× over IS 4 o [2], a cache-efficient version of the Samplesort and one of the fastest available sorting implementations [40].…”

Vortex: Extreme-Performance Memory Abstractions for Data-Intensive Streaming Applications

“…These results demonstrate that Vortex-S achieves a substantial improvement over the previous methods, while incurring negligible RAM overhead 𝜖. On Skylake-X (i.e., 𝑐 3 ), it beats the fastest in-place methods [42], [48] by 3 − 4× and STL quicksort by 11×. While Vortex-S is hands-down the fastest technique that can sort 24 GB of keys on these machines, it is interesting to see how its performance stacks up against the best outof-place methods.…”

Dr. Top-k: Delegate-Centric Top-k on GPUs

“…Consequently, pertinent top-𝑘 applications do not adopt priority queue-based top-𝑘. Instead, they use sort-and-choose approach for top-𝑘 computing on GPUs [6,18,33,44,48]. However, as shown in Figure 17, the GPU-based sort-and-choose top-𝑘 [6] takes much longer time than GPU-based top-k algorithms.…”