Robust Stratified Sampling Plans for Low Selectivity Queries

Joshi, Shantanu; Jermaine, Christopher

doi:10.1109/icde.2008.4497428

Cited by 25 publications

(12 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Density-Based Distribute Data Stream Clustering (DB-DDSC) determines clusters with different structures under the Big data stream environment [17]. Recruiting similar services in the same clusters provides collaborative recommendation services to the system [18]. Notably, regular research considers the frequency of instances as the primary metric in sampling [19 -21].…”

Section: Related Workmentioning

confidence: 99%

Analyzing Continuous Data Streams Using Improved Stratified Sampling and Ensemble Classification

Kathiresan¹,

Mohanta²,

Asari³

2018

IJIES

View full text Add to dashboard Cite

The streaming data technologies play a vital role in real-time applications. To analyze the data, Random sampling with replacement has a problem in drawing inferences from the small random sample, while sampling without replacement is not preferable to sub-streams that correspond to different sources. Hence, to effectively mine the data streams from heterogeneous sources, this work proposes Adaptive Reservoir sampling Of stream In a Time window (AdROIT) which partitions the streams in a window on time factor and determines the size of historical data on reference window regarding the data changes in the observation window. By measuring the standard deviation of the partitioned window, we can identify whether the changes in statistical properties of a data stream is due to one or multiple sources. The AdROIT allocates the reservoir sampling size to the source, ensures the adaptability, updates the ensemble classifier with dynamically estimated weight, decides accuracy of each member regarding weight. The experimental results show that the AdROIT provides better classification and mining results over heterogeneous data streams. The AdROIT increases the precision by 16%, compared to the Chain sampling under a high degree of heterogeneity. Under the same scenario, the proposed scheme increases the recall by 30 %, more than that in Chain sampling. In high degree of heterogeneity, the Chain sampling utilizes 40kb for storage, more than that of Chain sampling. Finally, the high window size reduces the execution time in AdROIT by 15 seconds and improves the recall by 40%, compared to the Chain sampling.

show abstract

Section: Related Workmentioning

confidence: 99%

Analyzing Continuous Data Streams Using Improved Stratified Sampling and Ensemble Classification

Kathiresan¹,

Mohanta²,

Asari³

2018

IJIES

View full text Add to dashboard Cite

show abstract

“…Because the data records are hidden under limited query interfaces in these systems, sampling involves very distinct challenges. Sampling for Aggregation Queries: Sampling algorithms have also been studied in the context of aggregation queries on large data bases [18], [1], [19], [25]. Approximate Pre-Aggregation (APA) [18] was proposed to estimate aggregation queries over categorical data utilizing precomputed statistics about the dataset.…”

Section: Related Workmentioning

confidence: 99%

“…Sampling for Aggregation Queries: Sampling algorithms have also been studied in the context of aggregation queries on large data bases [18], [1], [19], [25]. Approximate Pre-Aggregation (APA) [18] was proposed to estimate aggregation queries over categorical data utilizing precomputed statistics about the dataset. Wu et al [25] proposed a Bayesian method for guessing the extreme values in a dataset based on the learned query shape pattern and characteristics from previous workloads.…”

Section: Related Workmentioning

confidence: 99%

Stratified Sampling for Data Mining on the Deep Web

Liu

Wang

Agrawal

2010

2010 IEEE International Conference on Data Mining

View full text Add to dashboard Cite

In recent years, one mode of data dissemination has become extremely popular, which is the deep web. Like any other data source, data mining on the deep web can produce important insights or summary of results. However, data mining on the deep web is challenging because the databases cannot be accessed directly, and therefore, data mining must be performed based on sampling of the datasets. The samples, in turn, can only be obtained by querying the deep web databases with specific inputs.In this paper, we target two related data mining problems, which are association mining and differential rule mining. We develop stratified sampling methods to perform these mining tasks on a deep web source. Our contributions include a novel greedy stratification approach, which processes the query space of a deep web data source recursively, and considers both the estimation error and the sampling costs. We have also developed an optimized sample allocation method that integrates estimation error and sampling costs. Our experiment results show that our algorithms effectively and consistently reduce sampling costs, compared with a stratified sampling method that only considers estimation error. In addition, compared with simple random sampling, our algorithm has higher sampling accuracy and lower sampling costs.

show abstract

“…These include sequential sampling analysis [Haas and Swami, 1992;Hou et al, 1991], keeping additional statistics to improve the estimation [Haas and Swami, 1995], labelling the tuples and using label-dependent estimation procedures [Ganguly et al, 1996], or applying the cumulative distribution function inversion procedure [Wu et al, 2001]. Some work also looked at nonuniform sampling [Babcock et al, 2003;Estan and Naughton, 2006] and stratified sampling [Chaudhuri et al, 2007;Joshi and Jermaine, 2008]. Despite all these relevant contributions, online sampling is still considered too expensive for most applications.…”

Section: Related Workmentioning

confidence: 99%

The VC-Dimension of SQL Queries and Selectivity Estimation through Sampling

Riondato

Akdere

Çetintemel

et al. 2011

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

Abstract. We develop a novel method, based on the statistical concept of VC-dimension, for evaluating the selectivity (output cardinality) of SQL queries -a crucial step in optimizing the execution of large scale database and data-mining operations. The major theoretical contribution of this work, which is of independent interest, is an explicit bound on the VC-dimension of a range space defined by all possible outcomes of a collection (class) of queries. We prove that the VC-dimension is a function of the maximum number of Boolean operations in the selection predicate, and of the maximum number of select and join operations in any individual query in the collection, but it is neither a function of the number of queries in the collection nor of the size of the database. We develop a method based on this result: given a class of queries, it constructs a concise random sample of a database, such that with high probability the execution of any query in the class on the sample provides an accurate estimate for the selectivity of the query on the original large database. The error probability holds simultaneously for the selectivity estimates of all queries in the collection, thus the same sample can be used to evaluate the selectivity of multiple queries, and the sample needs to be refreshed only following major changes in the database. The sample representation computed by our method is typically sufficiently small to be stored in main memory. We present extensive experimental results, validating our theoretical analysis and demonstrating the advantage of our technique when compared to complex selectivity estimation techniques used in PostgreSQL and the Microsoft SQL Server.

show abstract

Robust Stratified Sampling Plans for Low Selectivity Queries

Cited by 25 publications

References 9 publications

Analyzing Continuous Data Streams Using Improved Stratified Sampling and Ensemble Classification

Analyzing Continuous Data Streams Using Improved Stratified Sampling and Ensemble Classification

Stratified Sampling for Data Mining on the Deep Web

The VC-Dimension of SQL Queries and Selectivity Estimation through Sampling

Contact Info

Product

Resources

About