Parallel evaluation of multi-join queries

Abstract

In the PRISMA project, a 100-node parallel machine was constructed by Philips Electronics and Dutch academia, along with a parallel main memory database sytem (PRISMA/DB). When this paper was written, the project had just finished (along the with idea of database machines). This paper abstracts from the particular features of PRISMA/DB, and evaluates and analyzes the performance trade-offs for a wide range of parallel query processing strategies. Its clear style of presentation, along with careful attention to previous work both in its discussion as well as in the experiments and analysis, make this paper into a concise introductory or "refereshment" text for researchers interested in parallel query execution. Currently, parallel databases may no longer be in vogue, but this line of work, in particular the non-blocking symmetric hash-join also described here, is nowadays frequently cited in articles on stream query processing and peer-to-peer databases. For me personally, a reminiscence of this paper automatically becomes a reminiscence of its primary author, Annita Wilschut, who is no longer active in our field after health problems. She was the coordinator of the (post-PRISMA) research project containing my Ph.D. track. Her sense of community and willingness to share knowledge, resulted in her teaching me to be a scientist, and enthusistically transmitting basic "database values", so crucial for our research field (e.g., the importance of abstract query languages that speciy "what" not "how", allowing both data independence and query optimization, etc. etc.). Ihab Ilyas, University of Waterloo, ilyas@uwaterloo.ca. [Ronald Fagin, Amnon Lotem and Moni Naor. Optimal aggregation algorithms for middleware. There are many papers that significantly influenced my research; hence, I decided to pick the paper with the most direct impact on my Ph.D. thesis. I became first aware of this paper by attending the best paper award presentation in PODS 2001, Santa Barbara. I still remember the lively presentation of the paper by Ronald Fagin. I was impressed by the fact that the algorithms are extremely intuitive and the optimality results are very strong. I read the paper during the conference and its extended version from Ronalds website, later on. I could immediately see a strong relationship to what I was doing at Purdue; I was working on providing efficient query processing techniques for top-k similarity queries, addressing the inefficiency of current query engines in handling this type of query, which is dominant in many applications. This paper introduced a family of optimal rank-aggregation algorithms for combining multiple ranked lists according to some monotone combining function. The objective was to get the overall top-k objects using the minimum number of accesses to the inputs lists. Reading the paper raised several research questions, one of which was how to leverage these simple and extremely efficient algorithms in relational databases to develop rank-aware query processing. In my Ph.D. thesis, I focused on ge...

The complexity of acyclic conjunctive queries

“…Our algorithms exploit inter-operation parallelism, that is, processing different primitive operations in parallel, minimizing the length of the sequence of parallel joins (while keeping the size of intermediate relations small). Note that the exploitation of inter-operation parallelism has been recognized as a relevant feature for parallel databases, and its importance has been confirmed also by concrete experiments [Wilschut et al 1995]. Intra-operation parallelism, that is, the parallel execution of a single primitive operation (e.g., join), is orthogonal and can be added easily to our model by parallelizing single join executions.…”

“…are treated as primitives. In such database models, there is a distinction between intra-operation parallelism, that is, the parallel execution of a single primitive operation (e.g., join), and inter-operation parallelism, that is, processing different primitive operations in parallel [Wilschut et al 1995]. We may assume that in a given parallel DBMS the issues of intra-operation parallelism are solved, that is, that a set of highly parallel procedures for performing single relational operations is given.…”

“… See, for example, DeWitt and Gray [1992],Hasan et al [1996],Hong and Stonebreaker [1993],Lakshmi and Yu [1990], andWilschut et al [1995].…”

ModParOpt: a Modular Query Optimizer for Multi-query Parallel Databases

“…We put this strategy at the second level, motivated by the works done on parallel in-memory databases showing excellent performances e.g. [24].…”

Adaptive use of a cluster of PCs for data warehousing applications

“…Faster processors and more efficient algorithms of query evaluation can help, but only to a certain extent. A number of techniques [11, 15,16] have been suggested by the researchers to speed up the query processing in OLAP. Some of these techniques Although, these methods do speed up the retrieval process of OLAP and data warehouse queries, they are still inadequate for dealing with vast amounts of data.…”