Dynamic query scheduling in data integration systems

Bouganim, Luc; Fabret, Françoise; Mohan, C.; Valduriez, Patrick

doi:10.1109/icde.2000.839442

Cited by 48 publications

(37 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…These systems focused on integrating independent sources, and did not consider the explicit controls, such as placement or replication, found in BigDAWG. We also will extend work on executing queries across several disjoint execution sites [5,10]. This includes the ability to decompose queries into subqueries for partial execution [17], and adaptive query processing to handle uncertainty in database performance [2].…”

Section: Related Workmentioning

confidence: 99%

The BigDAWG Polystore System

Duggan¹,

et al. 2015

View full text Add to dashboard Cite

This paper presents a new view of federated databases to address the growing need for managing information that spans multiple data models. This trend is fueled by the proliferation of storage engines and query languages based on the observation that "no one size fits all". To address this shift, we propose a polystore architecture; it is designed to unify querying over multiple data models. We consider the challenges and opportunities associated with polystores. Open questions in this space revolve around query optimization and the assignment of objects to storage engines. We introduce our approach to these topics and discuss our prototype in the context of the Intel Science and Technology Center for Big Data.

show abstract

Section: Related Workmentioning

confidence: 99%

The BigDAWG Polystore System

Duggan¹,

et al. 2015

View full text Add to dashboard Cite

show abstract

“…Several dynamic optimization methods [1,2,6,7,21,22,34] are proposed in the literature for correcting the sub-optimal execution plans. These methods are classified in four types (i) Replacement,(ii) Scheduling [15], (iii): Reoptimization [1,2,12], and (iv) Uses of dynamic operators [23,24].…”

Section: Related Workmentioning

confidence: 99%

“…These methods are classified in four types (i) Replacement,(ii) Scheduling [15], (iii): Reoptimization [1,2,12], and (iv) Uses of dynamic operators [23,24]. Each dynamic optimization method is able in adapting the execution plans in order to react to one or several events: (i) Estimation errors [15],(ii) Memory available [7,8,28,30], (iii): Delays in data arrival rates [1,2], and (iv) Users Preferences. The modification level of the executions plans can also be differ from a method to another.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Mobile Join Algorithms based on Mobiles Agents for Large Scale Distributed Query Optimization

Hussein¹

2012

IJAIS

View full text Add to dashboard Cite

In the large scale distributed environment, the query optimization presents new problems because of the data unavailability, the estimations inaccuracies and environment instability. In this paper, we address the sub-optimality of executions plans caused by these problems. We propose to extend the join algorithms based on mobile agents in order to correct the sub-optimality. This extension allows the join to change their execution site. Indeed, the mobile agent executing a join adapts to changes in characteristics of the execution environment (e.g. network bandwidth, available memory) and responds to the estimations inaccuracies (e.g. size of intermediate relations). The performance evaluation shows that the proposed algorithms improve the response time whatever the variation of estimations errors.

show abstract

“…These dynamic factors make query optimization in an MDBS even more challenging. A number of researchers have studied the issues on dynamic query optimization in an MDBS recently (Amsaleg, 1996;Bouganim, 2000;Ng, 1999;Urhan, 1998). All the dynamic techniques proposed so far have a common characteristic, that is, they modify or re-generate an execution plan for a given query based on dynamic information observed at run time.…”

Section: Introductionmentioning

confidence: 99%

Global Query Optimization Based on Multistate Cost Models for a Dynamic Multidatabase System

Zhu

Haridas

Hou

Enterprise Information Systems V

View full text Add to dashboard Cite

Abstract:Global query optimization in a multidatabase system (MDBS) is a challenging issue since some local optimization information such as local cost models may not be available at the global level due to local autonomy. It becomes even more difficult when dynamic environmental factors are taken into consideration. In our previous work, a qualitative approach was suggested to build so-called multistate cost models to capture the performance behavior of a dynamic multidatabase environment. It has been shown that a multistate cost model can give a good cost estimate for a query run in any contention state in the dynamic environment. In this paper, we present a technique to perform query optimization based on multistate cost models for a dynamic MDBS. Two relevant algorithms are proposed. The first one selects a set of representative system environmental states for generating an execution plan with multiple versions for a given query at compile time, while the second one efficiently determines the best version to invoke for the query at run time. Experiments demonstrate that the proposed technique is quite promising for performing global query optimization in a dynamic MDBS. Compared with related work on dynamic query optimization, our approach has an advantage of avoiding the high overhead for modifying or re-generating an execution plan for a query based on dynamic run-time information.

show abstract

Dynamic query scheduling in data integration systems

Cited by 48 publications

References 13 publications

The BigDAWG Polystore System

The BigDAWG Polystore System

Mobile Join Algorithms based on Mobiles Agents for Large Scale Distributed Query Optimization

Global Query Optimization Based on Multistate Cost Models for a Dynamic Multidatabase System

Contact Info

Product

Resources

About