Adapting to Changing Resource Performance in Grid Query Processing

Gounaris, Anastasios; Smith, Jim; Paton, Norman W.; Sakellariou, Rizos; Fernandes, Alvaro A. A.; Watson, Paul

doi:10.1007/11611950_4

Cited by 31 publications

(38 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As mentioned, we adapt a model previously designed to address query optimization problems [14,15], to address issues of record linkage.…”

Section: Exact and Approximate Join Op-eratorsmentioning

confidence: 99%

“…Current results from the adaptive query processing (AQP) framework [8] show how, in certain cases, the query processor can use these estimates to modify the query plan during execution, namely by replacing a physical operator with another that performs the same function [12,14,15,18,19,28,25]. This idea has proven viable for pipelined query plans [11], primarily as a dynamic optimization technique to improve the performance of a complex query, in cases where the initial query plan produced by the optimizer proves inefficient.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Time-completeness trade-offs in record linkage using adaptive query processing

Lengu

Missier

Fernandes

et al. 2009

Proceedings of the 12th International Conference on Extending Database Technology: Advances in Database Technology

Self Cite

View full text Add to dashboard Cite

Applications that involve data integration among multiple sources often require a preliminary step of data reconciliation in order to ensure that tuples match correctly across the sources. In dynamic settings such as data mashups, however, traditional offline data reconciliation techniques that require prior availability of the data may not be applicable. The alternative, performing similarity joins at query time, is computationally expensive, while ignoring the mismatch problem altogether leads to an incomplete integration. In this paper we make the assumption that, in some dynamic integration scenarios, users may agree to trade the completeness of a join result in return for a faster computation. We explore the consequences of this assumption by proposing a novel, hybrid join algorithm that involves a combination of exact and approximate join operators, managed using adaptive query processing techniques. The algorithm is optimistic: it can switch between physical join operators multiple times throughout query processing, but it only resorts to approximate join operators when there is statistical evidence that result completeness is compromised. Our experiments show that sensible savings in join execution time can be achieved in practice, at the expense of a modest reduction in result completeness.

show abstract

“…As mentioned, we adapt a model previously designed to address query optimization problems [14,15], to address issues of record linkage.…”

Section: Exact and Approximate Join Op-eratorsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Time-completeness trade-offs in record linkage using adaptive query processing

Lengu

Missier

Fernandes

et al. 2009

Proceedings of the 12th International Conference on Extending Database Technology: Advances in Database Technology

Self Cite

View full text Add to dashboard Cite

show abstract

“…Thus far, the emphasis has been on architectures for the execution of SQL-like queries that span multiple Web Services (e.g., [16,3,19]), wide area query optimization (e.g., [22,14]) and the associated resource scheduling decisions (e.g., [13]). However, an important factor is the optimization of the data transfer cost for WSs that encapsulate data sources.…”

Section: Introductionmentioning

confidence: 99%

Self-optimizing block transfer in web service grids

Gounaris

Yfoulis²,

Sakellariou

et al. 2007

Proceedings of the 9th Annual ACM International Workshop on Web Information and Data Management

Self Cite

View full text Add to dashboard Cite

Nowadays, Web Services (WSs) play an increasingly important role in Web data management solutions, since they offer a practical solution for accessing and manipulating data sources spanning administrative domains. Nevertheless, they are notoriously slow and transferring large data volumes across WSs becomes the main bottleneck in such WS-based applications. This paper deals with the problem of minimizing at runtime, in a self-managing way, the data transfer cost of a WS encapsulating a data source. To reduce the transfer cost, the data volume is typically divided into blocks. In this case, response time exhibits a quadratic-like, non-linear behavior with regards to the block size; as such, minimizing the transfer cost entails finding the optimum block size. This situation is encountered in several systems, such as WS Management Systems (WSMSs) for DBMS-like data management over wide area service-based networks, and WSs for accessing and integrating traditional DBMSs. The main challenges in this problem include (i) the unavailability of an analytical model; (ii) the presence of noise, which incurs local minima; (iii) the volatility of the environment, which results into a moving optimum operating point; and (iv) the requirements for fast convergence to the optimal size of the request from the side of the client rather than of the server, and for low overshooting. This paper presents two novel solutions for detecting the optimum block size during data transmission, thus yielding lower response times. The solutions are inspired by the broader areas of runtime optimization and switching extremum control. They incorporate heuristics to avoid local optimal points, and address all the afore-mentioned challenges. The effectiveness and efficiency of the solutions is verified through empirical evaluation in real cases.

show abstract

“…The plan makes explicit the decisions made by the query optimizer, e.g., with respect to the order of evaluation of operators, the algorithms and auxiliary data structures to be used, the allocation of plan fragments to resources, and the level of partitioned parallelism. An adaptive query processor (e.g., [5]) may revise any of these different kinds of decision at query runtime.…”

Section: Introductionmentioning

confidence: 99%

A Foundation for the Replacement of Pipelined Physical Join Operators in Adaptive Query Processing

Eurviriyanukul

Fernandes

Paton

2006

Current Trends in Database Technology – EDBT 2006

Self Cite

View full text Add to dashboard Cite

Abstract. Adaptive query processors make decisions as to the most effective evaluation strategy for a query based on feedback received while the query is being evaluated. In essence, any of the decisions made by the optimizer (e.g., on operator order or on which operators to use) may be revisited in an adaptive query processor. This paper focuses on changes to physical operators (e.g., the specific join operators used, such as hash-join or merge-join) in pipelined query evaluators. In so doing, the paper characterizes the runtime properties of pipelined operators in a way that makes explicit when specific operators may be replaced, and that allows the validity of operator replacements to be proved. This is illustrated with reference to the substitution of join operators during their evaluation.

show abstract

Adapting to Changing Resource Performance in Grid Query Processing

Cited by 31 publications

References 16 publications

Time-completeness trade-offs in record linkage using adaptive query processing

Time-completeness trade-offs in record linkage using adaptive query processing

Self-optimizing block transfer in web service grids

A Foundation for the Replacement of Pipelined Physical Join Operators in Adaptive Query Processing

Contact Info

Product

Resources

About