A Skew-Insensitive Algorithm for Join and Multi-join Operations on Shared Nothing Machines

Proceedings of the Third International Conference on Software and Data Technologies Special Session on Applications in Banking

2008

Abstract:Pipelined parallelism was largely studied and successfully implemented, on shared nothing machines, in several join algorithms in the presence of ideal conditions of load balancing between processors and in the absence of data skew. The aim of pipelining is to allow flexible resource allocation while avoiding unnecessary disk input/output for intermediate join results in the treatment of multi-join queries.The main drawback of pipelining in existing algorithms is that communication and load balancing remain limited to the use of static approaches (generated during query optimization phase) based on hashing to redistribute data over the network and therefore cannot solve data skew problem and load imbalance between processors on heterogeneous multi-processor architectures where the load of each processor may vary in a dynamic and unpredictable way. In this paper, we present a new parallel join algorithm allowing to solve the problem of data skew while guaranteeing perfect balancing properties, on heterogeneous multi-processor Shared Nothing architectures. The performance of this algorithm is analyzed using the scalable portable BSP (Bulk Synchronous Parallel) cost model.

Section: Resultsmentioning

confidence: 55%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Pipelined Parallelism in Multi-Join Queries on Heterogeneous Shared Nothing Architectures

Hassan

Proceedings of the Third International Conference on Software and Data Technologies Special Session on Applications in Banking

2008

“…For this reason, parallel processing of these queries is highly recommended in order to obtain acceptable response time (Bamha, 2005). Research has shown that join, which is one of the most expensive operations in DBMS, is parallelizable with near-linear speed-up only in ideal cases (Bamha and Hains, 2000). However, data skew degrades the performance of parallel systems (Bamha and Hains, 1999;Bamha and Hains, 2000;Seetha and Yu, 1990;Hua and Lee, 1991;Wolf et al, 1994;.…”

Section: Introductionmentioning

confidence: 99%

“…Research has shown that join, which is one of the most expensive operations in DBMS, is parallelizable with near-linear speed-up only in ideal cases (Bamha and Hains, 2000). However, data skew degrades the performance of parallel systems (Bamha and Hains, 1999;Bamha and Hains, 2000;Seetha and Yu, 1990;Hua and Lee, 1991;Wolf et al, 1994;. Thus, effective parallel algorithms that evenly distribute the load among processors and minimizes the inter-site communication must be employed in parallel and distributed systems in order to obtain acceptable performance.…”

Section: Introductionmentioning

confidence: 99%

Parallel Processing of “Group-By Join” Queries on Shared Nothing Machines

Hassan

Communications in Computer and Information Science

Self Cite

Abstract:SQL queries involving join and group-by operations are frequently used in many decision support applications. In these applications, the size of the input relations is usually very large, so the parallelization of these queries is highly recommended in order to obtain a desirable response time. The main drawbacks of the presented parallel algorithms that treat this kind of queries are that they are very sensitive to data skew and involve expensive communication and Input/Output costs in the evaluation of the join operation. In this paper, we present an algorithm that minimizes the communication cost by performing the group-by operation before redistribution where only tuples that will be present in the join result are redistributed. In addition, it evaluates the query without the need of materializing the result of the join operation and thus reducing the Input/Output cost of join intermediate results. The performance of this algorithm is analyzed using the scalable and portable BSP (Bulk Synchronous Parallel) cost model which predicts a near-linear speed-up even for highly skewed data.

An Efficient Equi-semi-join Algorithm for Distributed Architectures

Lecture Notes in Computer Science

Hains

2005

Self Cite

Semi-joins is the most used technique to optimize the treatment of complex relational queries on distributed architectures. However the overcost related to semi-joins computation can be very high due to data skew and to the high cost of communication in distributed architectures. In this paper we present a parallel equi-semi-join algorithm for shared nothing machines. The performance of this algorithm is analyzed using the BSP cost model and is proved to have asymptotic optimal complexity and perfect load balancing even for highly skewed data. This guarantees unlimited scalability in all situations for this key algorithm.