Practical selectivity estimation through adaptive sampling

Lipton, Richard J.; Naughton, Jeffrey F.; Schneider, Donovan A.

doi:10.1145/93597.93611

Cited by 210 publications

(92 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In practice, arbitrary data distributions are usually described using histograms, the idea of which is to divide the feature universe into a set of partitions such that the feature distribution of the objects within each partition is (almost) uniform [PiatetskyShapiro and Connell 1984;Liption et al 1990]. Figure 9 illustrates an example where four partitions A, B, C, D are allocated.…”

Section: General Temporal Datasetsmentioning

confidence: 99%

Cost models for overlapping and multiversion structures

Tao

Papadias

Zhang

2002

ACM Trans. Database Syst.

View full text Add to dashboard Cite

Overlapping and multiversion techniques are two popular frameworks that transform an ephemeral index into a multiple logical-tree structure in order to support versioning databases. Although both frameworks have produced numerous efficient indexing methods, their performance analysis is rather limited; as a result there is no clear understanding about the behavior of the alternative structures and the choice of the best one, given the data and query characteristics. Furthermore, query optimization based on these methods is currently impossible. These are serious problems due to the incorporation of overlapping and multiversion techniques in several traditional (e.g., financial) and emerging (e.g., spatiotemporal) applications. In this article, we reduce performance analysis of overlapping and multiversion structures to that of the corresponding ephemeral structures, thus simplifying the problem significantly. This reduction leads to accurate cost models that predict the sizes of the trees, the node/page accesses, and selectivity of queries. Furthermore, the models offer significant insight into the behavior of the structures and provide guidelines about the selection of the most appropriate method in practice. Extensive experimentation proves that the proposed models yield errors below 5 and 15% for uniform and nonuniform data, respectively.

show abstract

Section: General Temporal Datasetsmentioning

confidence: 99%

Cost models for overlapping and multiversion structures

Tao

Papadias

Zhang

2002

ACM Trans. Database Syst.

View full text Add to dashboard Cite

show abstract

“…Accordingly, we need some inexpensive technique to estimate f c and f r . We consider two main broadly used estimation approaches: estimation based on some default values and system statistics ( [25], [23]), and estimation based on sampling techniques ( [26], [27], [28]). Unlike the case with relational database systems, in string matching problems statistics about the collections of strings and how they interact with filters may not be available.…”

Section: B Cost Estimationmentioning

confidence: 99%

Approximate String Membership Checking: A Multiple Filter, Optimization-Based Approach

Sun

Naughton

Barman

2012

2012 IEEE 28th International Conference on Data Engineering

Self Cite

View full text Add to dashboard Cite

Abstract-We consider the approximate string membership checking (ASMC) problem of extracting all the strings or substrings in a document that approximately match some string in a given dictionary. To solve this problem, the current state-ofart approach involves first applying an approximate, fast filter, then applying a more expensive exact verification algorithm to the strings that pass the filter. Correspondingly, many string filters have been proposed. We note that different filters are good at eliminating different strings, depending on the characteristics of the strings in both the documents and the dictionary. We suspect that no single filter will dominate all other filters everywhere. Given an ASMC problem instance and a set of string filters, we need to select the optimal filter to maximize the performance. Furthermore, in our experiments we found that in some cases a sequence of filters dominates any of the filters of the sequence in isolation, and that the best set of filters and their ordering depend upon the specific problem instance encountered. Accordingly, we propose that the approximate match problem be viewed as an optimization problem, and evaluate a number of techniques for solving this optimization problem.

show abstract

“…The focus of this work is not on ad-hoc queries but to address scenarios such as query optimizer testing where it is necessary to obtain the cardinality-optimal plan. Finally, while sampling techniques has been used to obtain selectivity estimates during query optimization (e.g., [16]), it is not applicable for our problem since we are interested in obtaining exact cardinalities.…”

Section: Related Workmentioning

confidence: 99%

Exact cardinality query optimization for optimizer testing

Chaudhuri¹,

Narasayya²,

Ramamurthy³

2009

Proc. VLDB Endow.

View full text Add to dashboard Cite

The accuracy of cardinality estimates is crucial for obtaining a good query execution plan. Today"s optimizers make several simplifying assumptions during cardinality estimation that can lead to large errors and hence poor plans. In a scenario such as query optimizer testing it is very desirable to obtain the "best" plan, i.e., the plan produced when the cardinality of each relevant expression is exact. Such a plan serves as a baseline against which plans produced by using the existing cardinality estimation module in the query optimizer can be compared. However, obtaining all exact cardinalities by executing appropriate subexpressions can be prohibitively expensive. In this paper, we present a set of techniques that makes exact cardinality query optimization a viable option for a significantly larger set of queries than previously possible. We have implemented this functionality in Microsoft SQL Server and we present results using the TPC-H benchmark queries that demonstrate their effectiveness.

show abstract

Practical selectivity estimation through adaptive sampling

Cited by 210 publications

References 13 publications

Cost models for overlapping and multiversion structures

Cost models for overlapping and multiversion structures

Approximate String Membership Checking: A Multiple Filter, Optimization-Based Approach

Exact cardinality query optimization for optimizer testing

Contact Info

Product

Resources

About