Skew-insensitive parallel algorithms for relational join

Abstract

Join is the most important and expensive operation in relational databases. The parallel join operation is very sensitive to the presence of the data skew. In this paper, we present two new parallel join algorithms for coarse grained machines which work optimally in presence of arbitrary amount of data skew. The rst algorithm is sort-based and the second is hash-based. Both of these algorithms employ a preprocessing phase (prior to the redistribution phase) to equally partition the work among the processors. The proposed algorithms have been designed for memory resident-data. However, they can be extended to disk resident-data. These algorithms are shown to be theoretically as well as practically scalable. Experimental results are provided on the IBM SP-2.

A New Framework for Join Product Skew

“…In static algorithms it is assumed that adequate information on skewed data is known before the application of the algorithm. [1], [4] and [11] expose static algorithms. On the contrary, [2], [6] and [12] propose techniques and algorithms according to which data skew is detected and encountered dynamically at run time.…”

“…Using the notion of the splitting values stored in a split vector, virtual processor partitioning [4] assigns multiple range partitions instead of one to each processor. Finally, authors in [1] assign a work weight function to each join attribute value in order to generate partitions of nearly equal weight.…”

“…In this paper we address the issue of join product skew. Various techniques and algorithms have been proposed in the literature to handle this type of skew ( [1], [4], [11], [2], [6], [12]). We introduce the notion of frequency classes, whose definition is based on the product of frequencies of the join attribute values.…”