Analysis-driven Engineering of Comparison-based Sorting Algorithms on GPUs

Abstract

We study the relationship between memory accesses, bank conflicts, thread multiplicity (also known as over-subscription) and instruction-level parallelism in comparison-based sorting algorithms for Graphics Processing Units (GPUs). We experimentally validate a proposed formula that relates these parameters with asymptotic analysis of the number of memory accesses by an algorithm. Using this formula we analyze and compare several GPU sorting algorithms, identifying key performance bottlenecks in each one of them. Based on this analysis we propose a GPU-efficient multiway mergesort algorithm, GPU-MMS, which minimizes or eliminates these bottlenecks and balances various limiting factors for specific hardware.We realize an implementation of GPU-MMS and compare it to sorting algorithm implementations in state-of-the-art GPU libraries on three GPU architectures. Despite these library implementations being highly optimized, we find that GPU-MMS outperforms them by an average of 21% for random integer inputs and 14% for random key-value pairs.

Energy performance of parallel sort algorithms

“…This work in the energy consumption model of data collations using the described algorithms assumes the frequency of the main processor and the number of cycles needed to complete the task. Research on the relationship between memory accesses, bank conflicts, thread multiplicity, and instruction-level parallelism in comparison-based sorting algorithms for Graphics Processing Units (GPUs) are described in [11]. In [6], the implementation of image transformation algorithm on mobile devices using the approximate computation methods is described.…”