Depth estimation for ranking query optimization

Schnaitter, Karl; Spiegel, Joshua; Polyzotis, Neoklis

doi:10.1007/s00778-008-0124-z

Cited by 19 publications

(21 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The problem of optimizing individual top-k queries was considered in [16,29], and we leverage their cost estimation techniques. So far as we know, multiple query optimization has only been studied in the unranked context [26,30,35].…”

Section: Related Workmentioning

confidence: 99%

Sharing work in keyword search over databases

Jacob

2011

Proceedings of the 2011 ACM SIGMOD International Conference on Management of Data

View full text Add to dashboard Cite

An important means of allowing non-expert end-users to pose ad hoc queries -whether over single databases or data integration systems -is through keyword search. Given a set of keywords, the query processor finds matches across different tuples and tables. It computes and executes a set of relational sub-queries whose results are combined to produce the k highest ranking answers.Work on keyword search primarily focuses on single-database, single-query settings: each query is answered in isolation, despite possible overlap between queries posed by different users or at different times; and the number of relevant tables is assumed to be small, meaning that sub-queries can be processed without using cost-based methods to combine work. As we apply keyword search to support ad hoc data integration queries over scientific or other databases on the Web, we must reuse and combine computation. In this paper, we propose an architecture that continuously receives sets of ranked keyword queries, and seeks to reuse work across these queries. We extend multiple query optimization and continuous query techniques, and develop a new query plan scheduling module we call the ATC (based on its analogy to an air traffic controller). The ATC manages the flow of tuples among a multitude of pipelined operators, minimizing the work needed to return the top-k answers for all queries. We also develop techniques to manage the sharing and reuse of state as queries complete and input data streams are exhausted. We show the effectiveness of our techniques in handling queries over real and synthetic data sets.

show abstract

Section: Related Workmentioning

confidence: 99%

Sharing work in keyword search over databases

Jacob

2011

Proceedings of the 2011 ACM SIGMOD International Conference on Management of Data

View full text Add to dashboard Cite

show abstract

“…Clearly, this is an unrealistic assumption, as in general one would need to execute the entity package finder query in its entirety in order to determine these values. Thus, a realization of the Layered algorithm would call for an estimation D This estimation problem is strongly related to the Depth Estimation problem in the context of a rank-join algorithm ( [16]). Indeed, [16] proposes techniques that could be readily adapted for estimating D ′ i .…”

Section: Estimating DImentioning

confidence: 99%

“…Thus, a realization of the Layered algorithm would call for an estimation D This estimation problem is strongly related to the Depth Estimation problem in the context of a rank-join algorithm ( [16]). Indeed, [16] proposes techniques that could be readily adapted for estimating D ′ i . However, this adaptation necessitates a strong assumption; namely that sufficient statistics are known about the distribution of the scores of packages wrt.…”

Section: Estimating DImentioning

confidence: 99%

“…Moreover, the techniques presented therein cannot be efficiently adapted to our problem, as the resulting algorithm would rely on estimations with very high runtime and/or storage overhead. Such estimation problems, albeit in much simpler settings, are discussed in [16]. The techniques proposed in this work cannot be efficiently applied in the entity package finder setting; the reason is that document score aggregation introduces an added complexity for providing the requisite statistics for estimation.…”

Section: Related Workmentioning

confidence: 99%

“…It then uses e to probe the known associations table, R, and retrieve the packages containing e(line 14). For every such packages p, if the scores of all entities in p are known, Layered calculates the exact score of p (using F pkg ) (lines [16][17][18][19]. Layered also maintains the current top k packages in a priority queue, and the score of the current top-k'th package, minK .…”

mentioning

confidence: 99%

See 2 more Smart Citations

Ranking objects based on relationships and fixed associations

Angel

Chaudhuri

Das

et al. 2009

Proceedings of the 12th International Conference on Extending Database Technology: Advances in Database Technology

View full text Add to dashboard Cite

Text corpora are often enhanced by additional metadata which relate real-world entities, with each document in which such entities are discussed. Such relationships are typically obtained through widely available Information Extraction tools. At the same time, interesting known associations typically hold among these entities. For instance, a corpus might contain discussions on hotels, cities and airlines; fixed associations among these entities may include: airline A operates a flight to city C, hotel H is located in city C.A plethora of applications necessitate the identification of associated entities, each best matching a given set of keywords. Consider the sample query: Find a holiday package in a "pet-friendly" hotel, located in a "historical" yet "lively" city, with travel operated by an "economical" and "safe" airline. These keywords are unlikely to occur in the textual description of entities themselves, (e.g., the actual hotel name or the city name or the airline name). Consequently to answer such queries, one needs to exploit both relationships between entities and documents (e.g., keyword "pet-friendly" occurs in a document that contains an entity specifying a hotel name H), and the known associations between entities (e.g., hotel H is located in city C).In this work, we focus on the class of "entity package finder" queries outlined above. We demonstrate that existing techniques cannot be efficiently adapted to solve this problem, as the resulting algorithm relies on estimations with excessive runtime and/or storage overheads. We propose an efficient algorithm to process such queries, over large corpora. We devise early pruning and termination strategies, in the presence of joins and aggregations (executed on entities extracted from text), that do not depend on any estimates. Our analysis and experimental evaluation on real and synthetic data demonstrates the efficiency and scalability of our approach.

show abstract

Distributed top-k aggregation queries at large

Neumann

Bender

Michel

et al. 2009

Distrib Parallel Databases

View full text Add to dashboard Cite

Top-k query processing is a fundamental building block for efficient ranking in a large number of applications. Efficiency is a central issue, especially for distributed settings, when the data is spread across different nodes in a network. This paper introduces novel optimization methods for top-k aggregation queries in such distributed environments. The optimizations can be applied to all algorithms that fall into the frameworks of the prior TPUT and KLEE methods. The optimizations address three degrees of freedom: 1) hierarchically grouping input lists into top-k operator trees and optimizing the tree structure, 2) computing data-adaptive scan depths for different input sources, and 3) data-adaptive sampling of a small subset of input sources in scenarios with hundreds or thousands of query-relevant network nodes. All optimizations are based on a statistical cost model that utilizes local synopses, e.g., in the form of histograms, efficiently computed convolutions, and estimators based on order statistics. The paper presents comprehensive experiments, with three different real-life datasets and using the ns-2 network simulator for a packet-level simulation of a large Internet-style network.

show abstract

Depth estimation for ranking query optimization

Cited by 19 publications

References 17 publications

Sharing work in keyword search over databases

Sharing work in keyword search over databases

Ranking objects based on relationships and fixed associations

Distributed top-k aggregation queries at large

Contact Info

Product

Resources

About