Frequency-based load shedding over a data stream of tuples

Chang, Joong Hyuk; Kum, Hye‐Chung

doi:10.1016/j.ins.2009.01.014

Cited by 10 publications

(6 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, since the load now is smaller than the estimated capacity, it is not clear how much more data should be shed. We also approach this by trying to drop an additional x% (line 19), with x started as 1 % and increased following Eq. 5.…”

Section: The Aloma Algorithmmentioning

confidence: 99%

“…Regarding the other common questions related to load shedding, i.e., what to shed and where to shed, ALoMa uses a general, domain-independent method of applying random dropping evenly from the input of all queries in the class. Other works on these questions, such as those considering semantic dropping (e.g., [19,22,42]) and determining where in the query network to shed data to minimize semantic loss (e.g., [14,42]), can be trivially plugged in to replace the basic method ALoMa is using. Note that all these schemes need to know when and how much load to shed, which is answered by ALoMa.…”

Section: Extensibility Of Dilosmentioning

confidence: 99%

“…Instead of randomly dropping tuples, semantic models are used in [19,20,22,42] to increase the usefulness of the query results after shedding. Also related to this question, in [24,25,27,36,39] the authors propose methods to shed load other than simply discarding tuples from a query network.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Avoiding class warfare: managing continuous queries with differentiated classes of service

2015

View full text Add to dashboard Cite

Data stream management systems (DSMSs) offer the most effective solution for processing data streams by efficiently executing continuous queries (CQs) over the incoming data. CQs inherently have different levels of criticality and hence different levels of expected quality of service (QoS) and quality of data (QoD). Adhering to such expected QoS/QoD metrics is even more important in cases of multi-tenant data stream management services. In this work, we propose DILoS, a framework that, through prioritybased scheduling and load shedding, supports differentiated QoS and QoD for multiple classes of CQs. Unlike existing works that consider scheduling and load shedding separately, DILoS is a novel unified framework that exploits the synergy between scheduling and load shedding. We also propose ALoMa, a general, adaptive load manager that DILoS is built upon. By its design, ALoMa performs better than the state-ofthe-art alternatives in three dimensions: (1) it automatically tunes the headroom factor, (2) it honors the delay target, (3) it is applicable to complex query networks with shared operators. We implemented DILoS and ALoMa in our real DSMS prototype system (AQSIOS) and evaluate their performance for a variety of real and synthetic workloads. Our experimental evaluation of ALoMa verified its clear superiority over the This article enhances and extends preliminary work [37] that was presented in the SMDB'11 Workshop.state-of-the-art approaches. Our experimental evaluation of the DILoS framework showed that it (a) allows the scheduler and load shedder to consistently honor CQs' priorities, (b) significantly increases system capacity utilization by exploiting batch processing, and (c) enables operator sharing among query classes of different priorities while avoiding priority inversion, i.e., a lower-priority class never blocks a higherpriority one.

show abstract

Section: The Aloma Algorithmmentioning

confidence: 99%

Section: Extensibility Of Dilosmentioning

confidence: 99%

See 1 more Smart Citation

Avoiding class warfare: managing continuous queries with differentiated classes of service

2015

View full text Add to dashboard Cite

show abstract

“…The works on where to shed ([5], [6], [7], [8]) discuss how to distribute the shedding to different locations in the query network when CQs have different requirements on data accuracy. The works on what to shed increase the usefulness of the retained data after shedding by either considering data semantics (e.g., [5], [9], [10], [11]), or using other ways to shed load instead of completely discarding tuples (e.g., [4], [12], [13]). All these works on the where and what questions require that the amount of load to shed, i.e., the answer to the questions of when and how much to shed, is known and provided as an input.…”

Section: Introductionmentioning

confidence: 99%

Self-managing load shedding for data stream management systems

Pham

Chrysanthis

Labrinidis

2013

2013 IEEE 29th International Conference on Data Engineering Workshops (ICDEW)

View full text Add to dashboard Cite

“…Recently, many data mining methods [3,8,18,25] for a data stream have been actively proposed. A data stream is an ordered sequence of objects o 1 ,.…”

Section: Introductionmentioning

confidence: 99%