Learning to predict response times for online query scheduling

Macdonald, Craig; Tonellotto, Nicola; Ounis, Iadh

doi:10.1145/2348283.2348367

Cited by 70 publications

(115 citation statements)

References 25 publications

Supporting

Mentioning

111

Contrasting

Unclassified

Order By: Relevance

“…Dynamic pruning techniques such as WAND and BMW o er some relief as they o er the potential to safely skip the decompression of postings and the scoring of documents that cannot make the current top K. is makes the exact response time of a query di cult to predict, as not every posting in the postings lists will be decompressed and scored. Nevertheless recent work has considered making accurate predictions on the e ciency of a query, either in terms of absolute response time [29], or in terms of those queries with response times exceeding a threshold [19,21].…”

Section: Related Workmentioning

confidence: 99%

“…E ciency predictions facilitate a number of applications for ensuring e cient yet e ective retrieval -for instance, routing queries among busy replicated query shard servers [29]; selectively deploying multiple CPU cores for slow queries [19,21]; or adjusting the pruning aggressiveness or size of K for di erent queries [5,14,38]. Of these, the work of Tonello o et al [38] is among the most similar to ours, in that they vary the number of documents to be retrieved, K, as well as the pruning aggressiveness, before passing to a learning-to-rank re-ranking phase, based on the predicted execution time of the query.…”

Section: Related Workmentioning

confidence: 99%

“…e time spent processing a query in dynamic pruning strategies depends on several factors, the most important being: (i) the number of terms to be processed, (ii) the total number of postings to be processed and (iii) the relative importance of terms, i.e., their score contribution to the overall relevance of a document [29]. As shown in [19,21,29,38], by using statistics derived from terms and queries it is possible to estimate the query processing time.…”

Section: Predicting Complex Operator E Ciencymentioning

confidence: 99%

“…As shown in [19,21,29,38], by using statistics derived from terms and queries it is possible to estimate the query processing time. However, since we must deal with ephemeral posting lists, most of the statistics used in prior research are not applicable.…”

Section: Predicting Complex Operator E Ciencymentioning

confidence: 99%

“…Indeed, while recent work in query e ciency prediction has shown the possibility of estimating the execution time of a query prior to its processing [19,21,29,38], none of the existing work has considered the execution time of queries containing query operators that generate ephemeral posting lists, such as #syn, or #1. is makes it di cult to select among query rewriting strategies that use such operators, as their likely execution time is unknown. Hence, in this work, we study the cost of scoring ephemeral posting lists, and use these observations to de ne accurate query e ciency predictions for advanced query operators.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Efficient & Effective Selective Query Rewriting with Efficiency Predictions

Macdonald

Tonellotto

Ounis

2017

Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval

Self Cite

View full text Add to dashboard Cite

To enhance e ectiveness, a user's query can be rewri en internally by the search engine in many ways, for example by applying proximity, or by expanding the query with related terms. However, approaches that bene t e ectiveness o en have a negative impact on e ciency, which has impacts upon the user satisfaction, if the query is excessively slow. In this paper, we propose a novel framework for using the predicted execution time of various query rewritings to select between alternatives on a per-query basis, in a manner that ensures both e ectiveness and e ciency. In particular, we propose the prediction of the execution time of ephemeral (e.g., proximity) posting lists generated from uni-gram inverted index posting lists, which are used in establishing the permissible query rewriting alternatives that may execute in the allowed time. Experiments examining both the e ectiveness and e ciency of the proposed approach demonstrate that a 49% decrease in mean response time (and 62% decrease in 95th-percentile response time) can be a ained without signi cantly hindering the e ectiveness of the search engine.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Predicting Complex Operator E Ciencymentioning

confidence: 99%

Section: Predicting Complex Operator E Ciencymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Efficient & Effective Selective Query Rewriting with Efficiency Predictions

Macdonald

Tonellotto

Ounis

2017

Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval

Self Cite

View full text Add to dashboard Cite

show abstract

Query cost estimation in graph databases via emphasizing query dependencies by using a neural reasoning network

et al. 2023

Concurrency and Computation

View full text Add to dashboard Cite

SummaryWith the increasing complexity of graph queries, query cost estimation has become a key challenge in graph databases. Accurate estimation results are critical for database administrators or database management systems to perform query processing or optimization tasks. An efficient and accurate estimation model can improve the estimation quality and make the produced results credible. Although learning‐based methods have been applied in query cost estimation, most of them are directed at relational queries and cannot be directly used for graph queries. Furthermore, most estimation approaches focus on the correlations between predicates or columns. The dependencies between query schema and query filter conditions and the correlation between query schema are ignored. In this study, we construct a novel deep learning model composed of reasoning and retrieval processes that can accurately capture the potential logical relationships in graph queries. This solves the above problems to some extent. In addition, we propose a query estimation framework that divides the estimation task into query workload generation, training data collection, feature extraction and encoding, and estimation model construction. The results of the experiment on real‐world datasets show that our estimation model can improve the estimation quality and outperforms other compared deep learning models in terms of estimation accuracy.

show abstract