Reducing hardware hit by queries in web search engines

Mendoza, Marcelo; Gil-Costa, Verónica; Ferrarotti, Flavio

doi:10.1016/j.ipm.2016.04.008

Cited by 9 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their experimental result shows that ReDDE has slightly higher performance compared to CORI and CRCS. The learning based collection selection was proposed by Mendoza et al (2016), where the broker should broadcast the user query to all collections instead of building centralized sample database. Even though their methodology eliminates the collection size estimate and building of centralized sample database, but it increases computational overhead of the broker in the form of pruning non-relevant result.…”

Section: Related Workmentioning

confidence: 99%

Embedding based learning for collection selection in federated search

Garba

Khalid

Ullah

et al. 2020

DTA

View full text Add to dashboard Cite

PurposeThere have been many challenges in crawling deep web by search engines due to their proprietary nature or dynamic content. Distributed Information Retrieval (DIR) tries to solve these problems by providing a unified searchable interface to these databases. Since a DIR must search across many databases, selecting a specific database to search against the user query is challenging. The challenge can be solved if the past queries of the users are considered in selecting collections to search in combination with word embedding techniques. Combining these would aid the best performing collection selection method to speed up retrieval performance of DIR solutions.Design/methodology/approachThe authors propose a collection selection model based on word embedding using Word2Vec approach that learns the similarity between the current and past queries. They used the cosine and transformed cosine similarity models in computing the similarities among queries. The experiment is conducted using three standard TREC testbeds created for federated search.FindingsThe results show significant improvements over the baseline models.Originality/valueAlthough the lexical matching models for collection selection using similarity based on past queries exist, to the best our knowledge, the proposed work is the first of its kind that uses word embedding for collection selection by learning from past queries.

show abstract

Section: Related Workmentioning

confidence: 99%

Embedding based learning for collection selection in federated search

Garba

Khalid

Ullah

et al. 2020

DTA

View full text Add to dashboard Cite

show abstract

“…On the other hand, when query traffic is high, it is convenient to process several queries in batch . We used a Yahoo query log that corresponds to a 3‐month period in 2009. This log contains 2 109 198 distinct queries that correspond to 3 991 719 queries.…”

Section: Experiments With Batches Of Queriesmentioning

confidence: 99%

Performance evaluation of single vs. batch of queries on GPUs

Gaioso

Gil-Costa

Guardia

et al. 2019

Concurrency and Computation

View full text Add to dashboard Cite

Summary The WAND processing strategy is a dynamic pruning algorithm designed for large scale Web search engines where fast response to queries is a critical service. The WAND is used to reduce the amount of computation by scoring only documents that may become part of the top‐k document results. In this paper, we present two parallel strategies for the WAND algorithm and compare their performance on GPUs. In our first strategy (named size‐based), the posting lists are evenly partitioned among thread blocks. Our second strategy (named range‐based) partitions the posting lists according to document identifier intervals; thus, partitions may have different sizes. We also propose three threshold sharing policies, named Local, Safe‐R, and Safe‐WR, which emulate the WAND algorithm global pruning technique. We evaluated our proposals with different amounts of work, from short to extra‐large queries, using single query processing and batch of queries. Results show that the size‐based strategy reports the highest speedups but at the cost of low quality of results. The range‐based algorithm retrievals the exact top‐k documents and maintains a good speedup. Moreover, both strategies are capable of scaling as the amount of work is increased. In addition, there is no significant difference in the performance of the three threshold sharing policies.

show abstract

“…IR is an essential research area that pushes the development of information technologies and applications in many domains across the industry. IR-based systems are at the core of many search engines, supporting tasks such as query routing [2], spam filtering [3], multimedia retrieval [4], and user interest mining [5]. IR is also a fundamental building block of many content-based recommender systems [6].…”

Section: Introductionmentioning

confidence: 99%

Topic Models Ensembles for AD-HOC Information Retrieval

2021

Self Cite

View full text Add to dashboard Cite

Ad hoc information retrieval (ad hoc IR) is a challenging task consisting of ranking text documents for bag-of-words (BOW) queries. Classic approaches based on query and document text vectors use term-weighting functions to rank the documents. Some of these methods’ limitations consist of their inability to work with polysemic concepts. In addition, these methods introduce fake orthogonalities between semantically related words. To address these limitations, model-based IR approaches based on topics have been explored. Specifically, topic models based on Latent Dirichlet Allocation (LDA) allow building representations of text documents in the latent space of topics, the better modeling of polysemy and avoiding the generation of orthogonal representations between related terms. We extend LDA-based IR strategies using different ensemble strategies. Model selection obeys the ensemble learning paradigm, for which we test two successful approaches widely used in supervised learning. We study Boosting and Bagging techniques for topic models, using each model as a weak IR expert. Then, we merge the ranking lists obtained from each model using a simple but effective top-k list fusion approach. We show that our proposal strengthens the results in precision and recall, outperforming classic IR models and strong baselines based on topic models.

show abstract

Reducing hardware hit by queries in web search engines

Cited by 9 publications

References 17 publications

Embedding based learning for collection selection in federated search

Embedding based learning for collection selection in federated search

Performance evaluation of single vs. batch of queries on GPUs

Topic Models Ensembles for AD-HOC Information Retrieval

Contact Info

Product

Resources

About