A stratified approach to progressive approximate joins

Tok, Wee Hyong; Bressan, Stéphane; Lee, Mong Li

doi:10.1145/1353343.1353414

Cited by 11 publications

(4 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In relational streams, flush algorithms have been proposed to maximize the output rate or to generate a subset of results as early as possible [7][8][9][10]. We can apply their techniques on coarse-grained spilling in XML, which is spilling complete topmost elements to disk.…”

Section: Related Workmentioning

confidence: 99%

“…To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Articles from this volume were presented at The which pushes some fractions of data to disks temporarily, is employed in relational stream engines [7][8][9][10]. In this work, we now introduce "structure-based spilling", a spilling technique customized for XML streams by considering the partial spillage of complex XML elements.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Achieving high output quality under limited resources through structure-based spilling in XML streams

2010

View full text Add to dashboard Cite

Because of high volumes and unpredictable arrival rates, stream processing systems are not always able to keep up with input data -resulting in buffer overflow and uncontrolled loss of data. To produce eventually complete results, load spilling, which pushes some fractions of data to disks temporarily, is commonly employed in relational stream engines. In this work, we now introduce "structurebased spilling", a spilling technique customized for XML streams by considering the partial spillage of possibly complex XML elements. Such structure-based spilling brings new challenges. When a path is spilled, multiple paths may be affected. We analyze possible spilling effects on the query paths and how to execute the "reduced" query to produce partial results. To select the reduced query that maximizes output quality, we develop three optimization strategies, namely, OptR, OptPrune and ToX. We also examine the clean-up stage to guarantee that an entire result set is eventually generated by producing supplementary results. Our experimental study demonstrates that our proposed solutions consistently achieve higher quality results compared to the state-of-the-art techniques.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Achieving high output quality under limited resources through structure-based spilling in XML streams

2010

View full text Add to dashboard Cite

show abstract

“…Representative of this family are XJoin [17], PMJ [8], HMJ [4], RPJ [13], and the kinds of variations, e.g., see [2,15,3,12,14,16,7,10]. The progressive production of results requires query processing algorithms that can make the best use of main memory and utilize secondary storage cleverly even if one or both sources are blocked, so a fully pipelined results can still be outputted even with blocking sources.…”

Section: Introductionmentioning

confidence: 99%

“…Thus, it is suitable for the case when the users would not require a complete answer to their query but rather the representative sample as large as possible given the resources available. In addition to web-based query, non-blocking join algorithms are useful in such scenarios as parallel spatial join [12,11], skyline query over aggregated data [3,1], sensor [5], stream processing [13,3,16].…”

Section: Introductionmentioning

confidence: 99%

Progressive Hash-Merge Join Algorithm

Chen

Yang

et al. 2008

2008 IEEE Pacific-Asia Workshop on Computational Intelligence and Industrial Application

View full text Add to dashboard Cite

In data streams or web scenario at highly variable and unpredictable rates, most fast join algorithms to date rely upon shifting to external join stage as soon as possible when blocked in order to enhance efficiency. But they have trouble with the following issues: the limit of external join and practical executing time. Classical progressive two-way joining technique based on hash, however, fail to deliver acceptable performance in such a scenario where relatively short intermittent delay exists in the gross. We propose a new progressive join algorithm based on hash-merge for improving the query response time, separating one merging transaction into multi-subtask, whose transacting sizes rest with the interval time. Additionally, a refined replacement selection tree and a fine granularity timestamp are applied, which help to make use of finite memory and ensure correctness respectively. Theory and experimental results show that our technique delivers results significantly fast under both reliable and unreliable network.

show abstract

Load-Aware Shedding in Stream Processing Systems

Rivetti¹,

Busnel

Querzoni

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Load shedding is a technique employed by stream processing systems to handle unpredictable spikes in the input load whenever available computing resources are not adequately provisioned. A load shedder drops tuples to keep the input load below a critical threshold and thus avoid unbounded queuing and system trashing. In this paper we propose Load-Aware Shedding (LAS), a novel load shedding solution that, unlike previous works, does not rely neither on a pre-defined cost model nor on any assumption on the tuple execution duration. Leveraging sketches, LAS efficiently builds and maintains at runtime a cost model to estimate the execution duration of each tuple with small error bounds. This estimation enables a proactive load shedding of the input stream at any operator that aims at limiting queuing latencies while dropping as few tuples as possible. We provide a theoretical analysis proving that LAS is an (ε, δ)-approximation of the optimal online load shedder. Furthermore, through an extensive practical evaluation based on simulations and a prototype, we evaluate its impact on stream processing applications, which validate the robustness and accuracy of LAS.

show abstract

A stratified approach to progressive approximate joins

Cited by 11 publications

References 25 publications

Achieving high output quality under limited resources through structure-based spilling in XML streams

Achieving high output quality under limited resources through structure-based spilling in XML streams

Progressive Hash-Merge Join Algorithm

Load-Aware Shedding in Stream Processing Systems

Contact Info

Product

Resources

About