An Adaptive Multi-Objective Scheduling Selection Framework for Continuous Query Processing

Sutherland, Timothy M.; Pielech, Brad; Zhu, Yan; Ding, Luping; Rundensteiner, Elke A.

doi:10.1109/ideas.2005.9

Cited by 13 publications

(9 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In carrying out adaptations that are essentially the same as those supported by Flux, which was developed for use in continuous query processing, we note that many of the results from this paper carry over to continuous query systems, where the requirement to support different qualities of service through adaptation has also been identified (e.g. [56]). However, in seeking to apply results more widely, we note that the architecture used to run parallel queries has a significant impact on the issues that affect adaptation; as an example, adaptation in XPRS [21] may occur during operator execution, but the shared-memory model avoids need for operator state movement that has a significant impact in shared-nothing settings.…”

Section: Related Workmentioning

confidence: 97%

Utility-driven adaptive query workload execution

Paton¹,

Aragão²,

Fernandes³

2012

Future Generation Computer Systems

View full text Add to dashboard Cite

Workload management coordinates access to and use of shared computational resources; adaptive workload execution revises resource allocation decisions dynamically in response to feedback about the progress of the workload or the behavior of the resources. Where the workload contains or consists of database queries, adaptive query processing (AQP) changes the way in which a query is being evaluated while the query is running. In parallel environments, available adaptations may change the allocation of query fragments to a machine, for example to remove load imbalance or change the parallelism level. Most AQP strategies act on individual queries with the objective of reducing response times. However, where adaptations affect the usage of shared resources, or the principal goal is to meet quality of service targets rather than to minimize overall response times, locally beneficial decisions may have globally detrimental effects. This paper describes the use of utility functions to coordinate adaptations that assign resources to query fragments from multiple queries, and demonstrates how a common framework can be used to support different objectives, specifically to minimize overall query response times and to maximize the number of queries meeting quality of service goals. Experiments using simulation compare the use of utility functions with the more common heuristic control strategies, demonstrating situations in which significant benefits can be obtained.

show abstract

Section: Related Workmentioning

confidence: 97%

Utility-driven adaptive query workload execution

Paton¹,

Aragão²,

Fernandes³

2012

Future Generation Computer Systems

View full text Add to dashboard Cite

show abstract

“…Operator scheduling has been addressed in several research efforts [Urhan and Franklin 2001;Carney et al 2003;Babcock et al 2003;Hammad et al 2003;Sutherland et al 2005]. Urhan and Franklin [2001] propose the ratebased (RB) scheduling policy for scheduling operators within a single query to improve response time.…”

Section: Related Workmentioning

confidence: 99%

Algorithms and metrics for processing multiple heterogeneous continuous queries

Sharaf

Chrysanthis

Labrinidis

et al. 2008

ACM Trans. Database Syst.

View full text Add to dashboard Cite

The emergence of monitoring applications has precipitated the need for Data Stream Management Systems (DSMSs), which constantly monitor incoming data feeds (through registered continuous queries), in order to detect events of interest. In this article, we examine the problem of how to schedule multiple Continuous Queries (CQs) in a DSMS to optimize different Quality of Service (QoS) metrics. We show that, unlike traditional online systems, scheduling policies in DSMSs that optimize for average response time will be different from policies that optimize for average slowdown, which is a more appropriate metric to use in the presence of a heterogeneous workload. Towards this, we propose policies to optimize for the average-case performance for both metrics. Additionally, we propose a hybrid scheduling policy that strikes a fine balance between performance and fairness, by looking at both the average-and worst-case performance, for both metrics. We also show how our policies can be adaptive enough to handle the inherent dynamic nature of monitoring applications. Furthermore, we discuss how our policies can be efficiently implemented and extended to exploit sharing in optimized multi-query plans and multi-stream CQs. Finally, we experimentally show using real data that our policies consistently outperform currently used ones.

show abstract

“…Other efforts look into adapting the scheduling method used [62,63,64]. [62] uses machine learning unit to adapt systems parameters to improve the output latency.…”

Section: Related Workmentioning

confidence: 99%