Query optimization for parallel execution

Ganguly, Šumit; Hasan, Waqar; Krishnamurthy, Ravi

doi:10.1145/130283.130291

Cited by 153 publications

(120 citation statements)

References 6 publications

Supporting

Mentioning

118

Contrasting

Unclassified

Order By: Relevance

“…They differ from their centralized counterparts in that the communication cost must be also considered. The adoption of parallelism during query execution makes the problem of query optimization even harder [27]. This has leaded to the adoption of more sophisticated dynamic programming techniques (e.g., [28]) or heuristics.…”

Section: Related Workmentioning

confidence: 99%

Optimization of decentralized multi-way join queries over pipelined filtering services

2012

View full text Add to dashboard Cite

The capability to optimize and execute complex queries over multiple remote services (e.g., web services) is of high significance for efficient data management in large scale distributed computing infrastructures, such as those enabled by grid and cloud computing technology. In this work, we investigate the optimization of queries that involve multiple data resources, each of which is processed by non-overlapping sets of remote filtering services. The main novelty of this work is the proposal of optimization algorithms that produce an ordering of calls to services so that the query response time is minimized. The distinctive features of our work lie in the consideration of direct, heterogenous links among services and of multiple data resources. To the best of our knowledge, there is no known algorithm for this problem and the evaluation results show that the proposed algorithms can yield significant performance improvements compared to naive approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

Optimization of decentralized multi-way join queries over pipelined filtering services

2012

View full text Add to dashboard Cite

show abstract

“…In a federation, costs must be decoupled into multiple dimensions under the control of various administrators. One proposal for a universal cost metric is hard currency [45], but typically there are other costs that are valuable to expose orthogonally, including response time [17], data freshness [36], and accuracy of computations [5].…”

Section: Decoupling Of Cost Factorsmentioning

confidence: 99%

“…As has been pointed out previously [17,48], optimizing for such an optimization goal requires the use of partial order dynamic programming technique. This technique is a generalization of the classical dynamic programming algorithm where the cost of each plan is computed as a vector and two costs are considered incomparable if neither is less than or equal to the other in all the dimensions 2 .…”

Section: Response Time Optimization Vs Total Cost Optimizationmentioning

confidence: 99%

Decoupled query optimization for federated database systems

Deshpande

Hellerstein

Proceedings 18th International Conference on Data Engineering

View full text Add to dashboard Cite

show abstract

“…As with other exhaustive search algorithms DP has an exponential worst case running time and space complexity of O(3 n ) and O(2 n ) respectively [14,22]. Let C(p) denote the cost of a subplan rooted at the QEP node p. The principle of optimality states that if two plans differ only by a single subplan rooted at p then the plan with the lowest C(p) will also have a lower complete plan cost [4]. The DP algorithm uses the principle of optimality to build the optimal plan by considering joining optimal subplans in a bottom up fashion.…”

Section: Enumeration Algorithmsmentioning

confidence: 99%

“…Resource contention refers to the situation where multiple processes wish to access the same resource. Data dependencies exist when an operator is required to wait for its input operators to complete before it is able to proceed [4]. As mentioned previously, in a centralized setting the response time of a given execution plan can be estimated by considering resource consumption i.e disk useage.…”

Section: Establishing Query Execution Plan Costmentioning

confidence: 99%

Query Optimization in Distributed Database Systems

Hurley¹

SpringerReference

View full text Add to dashboard Cite

AcknowledgmentsI would like to thank my supervisor Dr Dan Olteanu for his incredible level of enthusiasm and encouragement throughout the project. I am also very grateful for the continuous level of feedback and organisation as well as the amount of time he has devoted to answering my queries. I feel that I now approach complex and unknown problems with enthusiasm instead of apprehension as I used to. I couldn't have had a better supervisor. AbstractThe query optimizer is widely considered to be the most important component of a database management system. It is responsible for taking a user query and searching through the entire space of equivalent execution plans for a given user query and returning the execution plan with the lowest cost. This plan can then be passed to the executer, which can carry out the query. Plans can vary significantly in cost therefore it is important for the optimizer to avoid very bad plans. In this thesis we consider queries in positive relational algrebra form involving the conjunction of projections, selections and joins.The query optimization problem faced by everyday query optimizers gets more and more complex with the ever increasing complexity of user queries. The NP-hard join ordering problem is a central problem that an optimizer must deal with in order to produce optimal plans. Fairly small queries, involving less than 10 relations, can be handled by existing algorithms such as the classic Dynamic Programming optimization algorithm. However, for complex queries or queries involving multiple execution sites in a distributed setting the optimization problem becomes much more challenging and existing optimization algorithms find it difficult to cope with the complexity.In this thesis we present a cost model that allows inter-operator parallelism opportunities to be identified within query execution plans. This allows the response time of a query to be estimated more accurately. We merge two existing centralized optimization algorithms DPccp and IDP1 to create a practically more efficient algorithm IDP1ccp. We propose the novel Multilevel optimization algorithm framework that combines heuristics with existing centralized optimization algorithms. The distributed multilevel optimization algorithm (DistML) proposed in this paper uses the idea of distributing the optimization phase across multiple optimization sites in order to fully utilize the available system resources.

show abstract

Query optimization for parallel execution

Cited by 153 publications

References 6 publications

Optimization of decentralized multi-way join queries over pipelined filtering services

Optimization of decentralized multi-way join queries over pipelined filtering services

Decoupled query optimization for federated database systems

Query Optimization in Distributed Database Systems

Contact Info

Product

Resources

About