Top-k Query Evaluation with Probabilistic Guarantees

Abstract

Top-k queries based on ranking elements of multidimensional datasets are a fundamental building block for many kinds of information discovery. The best known general-purpose algorithm for evaluating top-k queries is Fagin's threshold algorithm (TA). Since the user's goal behind top-k queries is to identify one or a few relevant and novel data items, it is intriguing to use approximate variants of TA to reduce run-time costs. This paper introduces a family of approximate top-k algorithms based on probabilistic arguments. When scanning index lists of the underlying multidimensional data space in descending order of local scores, various forms of convolution and derived bounds are employed to predict when it is safe, with high probability, to drop candidate items and to prune the index scans. The precision and the efficiency of the developed methods are experimentally evaluated based on a large Web corpus and a structured data collection.

Top-k Linked Data Query Processing

“…The idea of pruning has been pursued by approximate top-k selection [18] approaches. However, we do not approximate, but only prune those partial results that are guaranteed not to be part of the final top-k results.…”

“…However, [18] addressed the selection top-k problem, which is different to our top-k join problem. More importantly, we do not rely on probabilistic estimates for pruning, but employ accurate upper bounds.…”

“…Similar to our pruning approach, [18] estimates the likelihood of partial results contributing to a final result (if the estimate is below a given threshold partial results are pruned). However, [18] addressed the selection top-k problem, which is different to our top-k join problem.…”

Exploring a Few Good Tuples from Text Databases

“…However, E -Upper can be time-consuming under some scenarios where we need to process a relatively large number of documents to lower the upperbound of a tuple that does not belong to the answer. This observation has been made in the topk processing case as well, leading to developing probabilistic algorithms [20]. Unfortunately, existing probabilistic algorithms typically require some apriori knowledge about the score distribution of the tuples.…”

“…In this paper, we adapted the generic Upper algorithm to our setting as discussed in Section IV-A, and we established the feasibility of our adaptation at processing good(k, ) queries, as discussed in Section VI. For processing top-k algorithms, a variety of probabilistic algorithms have also been explored [20], which exploit some a priori knowledge on the score distribution.…”