Distributed join algorithms on thousands of cores

Abstract

Traditional database operators such as joins are relevant not only in the context of database engines but also as a building block in many computational and machine learning algorithms. With the advent of big data, there is an increasing demand for efficient join algorithms that can scale with the input data size and the available hardware resources.In this paper, we explore the implementation of distributed join algorithms in systems with several thousand cores connected by a low-latency network as used in high performance computing systems or data centers. We compare radix hash join to sort-merge join algorithms and discuss their implementation at this scale. In the paper, we explain how to use MPI to implement joins, show the impact and advantages of RDMA, discuss the importance of network scheduling, and study the relative performance of sorting vs. hashing. The experimental results show that the algorithms we present scale well with the number of cores, reaching a throughput of 48.7 billion input tuples per second on 4,096 cores.

Design and Evaluation of an RDMA-aware Data Shuffling Operator for Parallel Database Systems

“…The shuffling operator is the only operator that will transmit and receive data over the network. Slow networks can be a bottleneck for parallel database systems [33] and data shuffling has been shown to be a significant contributor to the end-to-end query response time [2,3,37].…”

“…Still, about 30% of the cycles are idle and would be devoted to other activities in a well-designed database system. Barthels et al [2] Figure 10(b) MESQ/SR vs. IPoIB with 16 nodes in the FDR cluster). As also seen in the repartition pattern, the MESQ/SR algorithm shows good scalability in the FDR cluster while the MQ algorithms degrade.…”

“…We contribute six designs of the data shuffling operator that represent different trade-offs between (1) the number of open connections, (2) the contention for the shared network interface, (3) the RDMA transport function, and (4) how much memory should be reserved to shuffle data between nodes during query processing. We adopt the popular pull-based operator interface to permit database systems implementors to use the proposed techniques without radically redesigning their existing analytical processing engines.…”

Reproducible Floating-Point Aggregation in RDBMSs

“…For the case where the result is larger than a private cache, but smaller than the combined shared cache of all threads, Cieslewicz and Ross [11] show that SHAREDAGGREGATION may be a better solution than the other two, which uses uses a shared (lock-free) hash table, at least in the absence of skew. Similar techniques have been proposed for JOIN and SORT operators [4,5,6,7,9]. As we show in this paper, these techniques alone are not sufficient for reproducible floating-point numbers.…”

“…As a first solution for reproducible floating-point aggregation with GROUPBY, we propose a data type that can be used as drop-in replacement for intermediate aggregates of floatingpoint numbers in any state-of-the-art aggregation algorithm with little to no modification. 7 We base this type on the…”