Reservoir Sampling over Memory-Limited Stream Joins

Al-Kateb, Mohammed; Lee, Byung Suk; Wang, X.S.

doi:10.1109/ssdbm.2007.40

Cited by 5 publications

(7 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They also proposed an efficient way of maintaining the uniformity confidence above the lower limit for join queries in the stream environment. 17 Ting 18 proposed a sampling technique that satisfies the constraint conditions such as memory and sample amount and uses the uniformity confidence as a measure for evaluating the sampling algorithm. The uniformity confidence is a measure of how much data the sampling algorithm considers in the limited memory space.…”

Section: Uniformity Confidencementioning

confidence: 99%

“…Sampling range increase Continuous degradation of uniformity confidence smaller than (p 3 wLength), we increase the sample size by one to maintain the fixed sampling ratio p, and add the current incoming element to the sample along with the random value (Lines 11-13). On the other hand, if the current sample size is not larger than (p 3 wLength), we compare the random value of the current element with the element having the smallest random value in the sample list to check whether we replace the smallest one with the current one (Lines [15][16][17][18][19]. We repeat this process until the stream input ends or the user stops sampling (Lines 4-19).…”

Section: Past Sample Invariancementioning

confidence: 99%

See 1 more Smart Citation

Variable size sampling to support high uniformity confidence in sensor data streams

Kim

Gil

Moon

et al. 2018

International Journal of Distributed Sensor Networks

View full text Add to dashboard Cite

In order to rapidly process large amounts of sensor stream data, it is effective to extract and use samples that reflect the characteristics and patterns of the data stream well. In this article, we focus on improving the uniformity confidence of KSample, which has the characteristics of random sampling in the stream environment. For this, we first analyze the uniformity confidence of KSample and then derive two uniformity confidence degradation problems: (1) initial degradation, which rapidly decreases the uniformity confidence in the initial stage, and (2) continuous degradation, which gradually decreases the uniformity confidence in the later stages. We note that the initial degradation is caused by the sample range limitation and the past sample invariance, and the continuous degradation by the sampling range increase. For each problem, we present a corresponding solution, that is, we provide the sample range extension for sample range limitation, the past sample change for past sample invariance, and the use of UC-window for sampling range increase. By reflecting these solutions, we then propose a novel sampling method, named UC-KSample, which largely improves the uniformity confidence. Experimental results show that UC-KSample improves the uniformity confidence over KSample by 2.2 times on average, and it always keeps the uniformity confidence higher than the user-specified threshold. We also note that the sampling accuracy of UC-KSample is higher than that of KSample in both numeric sensor data and text data. The uniformity confidence is an important sampling metric in sensor data streams, and this is the first attempt to apply uniformity confidence to KSample. We believe that the proposed UC-KSample is an excellent approach that adopts an advantage of KSample, dynamic sampling over a fixed sampling ratio, while improving the uniformity confidence.

show abstract

Section: Uniformity Confidencementioning

confidence: 99%

Section: Past Sample Invariancementioning

confidence: 99%

Variable size sampling to support high uniformity confidence in sensor data streams

Kim

Gil

Moon

et al. 2018

International Journal of Distributed Sensor Networks

View full text Add to dashboard Cite

show abstract

“…quantile [10], heavy hitters [13] and distinct counts [9]) and join queries [7,8,1]. In this paper, we focus on progressive, approximate join queries where the results are streamed out to the user as soon as they are produced.…”

Section: Related Workmentioning

confidence: 99%

“…Several progressive approximate join algorithms [7,8,1] have been proposed for data stream applications. In [7] and its extended version [8], the motivation was the maximization of the result subset produced.…”

Section: Progressive Approximate Joinsmentioning

confidence: 99%

“…We quantify the notion of result quality, and propose a technique that is able to deliver good-quality results progressively. Though [1] also studied the use of reservoir sampling over memory-limited join, the focus of the work was on how to balance between the memory allocated for join buffers and the reservoir. In addition, [1] do not progressively output results.…”

Section: Progressive Approximate Joinsmentioning

confidence: 99%

See 1 more Smart Citation

A stratified approach to progressive approximate joins

Tok

Bressan

Lee

2008

Proceedings of the 11th International Conference on Extending Database Technology: Advances in Database Technology

View full text Add to dashboard Cite

Users often do not require a complete answer to their query but rather only a sample. They expect the sample to be either the largest possible or the most representative (or both) given the resources available. We call the query processing techniques that deliver such results 'approximate'. Processing of queries to streams of data is said to be 'progressive' when it can continuously produce results as data arrives. In this paper, we are interested in the progressive and approximate processing of queries to data streams when processing is limited to main memory. In particular, we study one of the main building blocks of such processing: the progressive approximate join. We devise and present several novel progressive approximate join algorithms. We empirically evaluate the performance of our algorithms and compare them with algorithms based on existing techniques. In particular we study the trade-off between maximization of throughput and maximization of representativeness of the sample.

show abstract

An Agile Sample Maintenance Approach for Agile Analytics

Zhang

et al. 2020

2020 IEEE 36th International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

Reservoir Sampling over Memory-Limited Stream Joins

Abstract: In

Cited by 5 publications

References 32 publications

Variable size sampling to support high uniformity confidence in sensor data streams

Variable size sampling to support high uniformity confidence in sensor data streams

A stratified approach to progressive approximate joins

An Agile Sample Maintenance Approach for Agile Analytics

Contact Info

Product

Resources

About