Algorithms and metrics for processing multiple heterogeneous continuous queries

Sharaf, Mohamed A.; Chrysanthis, Panos K.; Labrinidis, Alexandros; Pruhs, Kirk

doi:10.1145/1331904.1331909

Cited by 56 publications

(37 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multiple scheduling algorithms have been developed, aiming at guaranteeing the quality of service or quality of data, e.g., [16], or satisfactory performance, e.g., [14], or minimizing data staleness, e.g., [8].…”

Section: Related Workmentioning

confidence: 99%

RTDW-bench: Benchmark for Testing Refreshing Performance of Real-Time Data Warehouse

Jedrzejczak

Koszlajda

Wrembel

2012

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. In this paper we propose a benchmark, called RTDW-bench, for testing a performance of a real-time data warehouse. The benchmark is based on TPC-H. In particular, RTDW-bench permits to verify whether an already deployed RTDW is able to handle without any delays a transaction stream of a given arrival rate. The benchmark also includes an algorithm for finding the maximum stream arrival rate that can be handled by a RTDW without delays. The applicability of the proposed benchmark was verified in a RTDW implemented in Oracle11g.

show abstract

Section: Related Workmentioning

confidence: 99%

RTDW-bench: Benchmark for Testing Refreshing Performance of Real-Time Data Warehouse

Jedrzejczak

Koszlajda

Wrembel

2012

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…There has also been work on scheduling in real-time databases [1,10], but the focus was on scheduling queries to meet some quality-of-service requirements (e.g., deadline or response time), not updates. Similarly, related work on scheduling in Data Stream Management Systems only deals with scheduling query operators, e.g., to maximize throughput or minimize memory usage [2,5,9,14].…”

Section: Related Workmentioning

confidence: 99%

Scheduling to minimize staleness and stretch in real-time data warehouses

Bateni

Golab

Hajiaghayi

et al. 2009

Proceedings of the Twenty-First Annual Symposium on Parallelism in Algorithms and Architectures

View full text Add to dashboard Cite

We study scheduling algorithms for loading data feeds into real time data warehouses, which are used in applications such as IP network monitoring, online financial trading, and credit card fraud detection. In these applications, the warehouse collects a large number of streaming data feeds that are generated by external sources and arrive asynchronously. Data for each table are generated at a constant rate, different tables possibly at different rates. For each data feed, the arrival of new data triggers an update that appends the new data to the corresponding table; if multiple updates are pending for the same table, they are batched together before being loaded. At time τ , if a table has been updated with information up to time r ≤ τ , its staleness is defined as τ − r.Our first objective is to schedule the updates on one or more processors in a way that minimizes the total staleness. In order to ensure fairness, our second objective is to limit the maximum "stretch", which we define (roughly) as the ratio between the duration of time an update waits till it is finished being processed, and the length of the update.In contrast to earlier work proving the nonexistence of constant-competitive algorithms for related scheduling problems, we prove that any online nonpreemptive algorithm, no processor of which is ever voluntarily idle, incurs a staleness at most a constant factor larger than an obvious lower bound on total staleness (provided that the processors are sufficiently fast). We give a constant-stretch algorithm, provided that the processors are sufficiently fast, for the quasiperiodic model, in which tables can be clustered into a few groups such that the update frequencies within each group * Work done while the author was visiting AT&T LabsResearch.Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SPAA '09, August 11-13, 2009, Calgary, Alberta, Canada. Copyright 2009 vary by at most a constant factor. Finally, we show that our constant-stretch algorithm is also constant-competitive (subject to the same proviso on processor speed) in the quasiperiodic model with respect to total weighted staleness, where tables are assigned weights that reflect their priorities.

show abstract

“…in [15]) are less well suited for our work as they potentially "hide" waiting times of important elements.…”

Section: Prioritized Elements Of a Data Streammentioning

confidence: 99%

“…The efficiency of data stream processing and the latency of results highly depends on the scheduling strategy [15]. Several research papers address scheduling topics in DSMS, like the scheduling approaches in [3], Aurora [7], in NiagaraCQ [10] and in PIPES [6], to name a few.…”

Section: Related Workmentioning

confidence: 99%

A physical operator algebra for prioritized elements in data streams

et al. 2009

View full text Add to dashboard Cite

Data stream management systems are a natural choice to efficiently process continuous queries over high volume data streams, e.g., to monitor sensor data or transaction streams. An immediate reaction on detected critical or security relevant situations is essential for a secure and economic operation, as in our scenario of monitoring decentralized energy systems, which realize geographically distributed energy generation processes. Without further provisions existing processing approaches may lead to a delay of critical or security relevant messages in high load situations, e.g., caused by bursts.One way to allow an adequate processing in such situations is to prioritize queries that handle critical situations. Unfortunately, problems are not always solely identifiable by a query. Sometimes certain -e.g., out of range -data values or error messages indicate situations, which urge a faster processing of all queries processing these data. Traditional approaches on continuous query execution assume a stream order, typically based on timestamps, and a processing following this order. In this article we consider the prioritization of those elements and propose an out-of-order execution in the data stream.We provide a comprehensive and formally founded approach for prioritizing data stream elements. Prioritized elements benefit twice from our approach. On the one hand, they are able to "overtake" lower prioritized elements, e.g., in queues. On the other hand, prioritized results can be produced earlier in stateful operators than this would be possible in other approaches. Still, the semantics of the queries remains unchanged. We implemented our approach and show with measurements that a very low latency of prioritized elements can be achieved -even under high load. As a result, all queries that process prioritized elements can benefit from our approach.

show abstract

Algorithms and metrics for processing multiple heterogeneous continuous queries

Cited by 56 publications

References 26 publications

RTDW-bench: Benchmark for Testing Refreshing Performance of Real-Time Data Warehouse

RTDW-bench: Benchmark for Testing Refreshing Performance of Real-Time Data Warehouse

Scheduling to minimize staleness and stretch in real-time data warehouses

A physical operator algebra for prioritized elements in data streams

Contact Info

Product

Resources

About