Sync/Async parallel search for the efficient design and construction of web search engines

Gil-Costa, Verónica; Bonacic, Carolina; Baeza-Yates, Ricardo; Scherson, Isaac D.

doi:10.1016/j.parco.2010.02.001

Cited by 21 publications

(11 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In all cases we show results normalized in order to (1) better illustrate the comparative performance among them and (2) present a global view of comparative performance across several figures covering all datasets tested [36,34,41]. To this end, in each figure, we divide all the experimental times by the observed maximum in the particular experiment.…”

Section: Methodsmentioning

confidence: 99%

Distributed text search using suffix arrays

Arroyuelo¹,

Bonacic²,

Gil-Costa³

et al. 2014

Parallel Computing

Self Cite

View full text Add to dashboard Cite

Text search is a classical problem in Computer Science, with many data-intensive applications. For this problem, suffix arrays are among the most widely known and used data structures, enabling fast searches for phrases, terms, substrings and regular expressions in large texts. Potential application domains for these operations include large-scale search services, such as Web search engines, where it is necessary to efficiently process intensive-traffic streams of on-line queries. This paper proposes strategies to enable such services by means of suffix arrays. We introduce techniques for deploying suffix arrays on clusters of distributed-memory processors and then study the processing of multiple queries on the distributed data structure. Even though the cost of individual search operations in sequential (non-distributed) suffix arrays is low in practice, the problem of processing multiple queries on distributed-memory systems, so that hardware resources are used efficiently, is relevant to services aimed at achieving high query throughput at low operational costs. Our theoretical and experimental performance studies show that our proposals are suitable solutions for building efficient and scalable on-line search services based on suffix arrays. IntroductionIn the last decade, the design of efficient data structures and algorithms for textual databases and related applications has received a great deal of attention, due to the rapid growth of the amount of text data available from different sources. Typical applications support text searches over big text collections in a client-server fashion, where the user queries are answered by a dedicated server [15]. The server efficiency-in terms of running time-is of paramount importance in cases where the services demanded by clients generate a heavy work load. A feasible way to overcome the limitations of sequential computers is to resort to the use of several computers, or processors, which work together to serve the ever increasing client demands [19].One such approach to efficient parallelization is to distribute the data onto the processors, in such a way that it becomes feasible to exploit locality via parallel processing of user requests, each on a subset of the data. As opposed to shared-memory models, this distributed-memory model provides the benefit of better * Corresponding author. Address: Av. España 1680, Valparaíso, Chile. Phone: +56 2 432 6722. Fax: +56 2 432 6702. , in distributed memory systems, and describes strategies to reduce the inter-processor communication and to improve the load balance at search time. Indexed Text SearchingThe advent of powerful processors and cheap storage has enabled alternative models for information retrieval, other than the traditional one of a collection of documents indexed by a fixed set of keywords. One is the full text model, in which the user expresses its information need via words, phrases or patterns to be matched for, and the information system retrieves those documents containing the user-specified pa...

show abstract

Section: Methodsmentioning

confidence: 99%

Distributed text search using suffix arrays

Arroyuelo¹,

Bonacic²,

Gil-Costa³

et al. 2014

Parallel Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…When query traffic is low, it is convenient to process one query at a time. On the other hand, when query traffic is high, it is convenient to process several queries in batch . We used a Yahoo query log that corresponds to a 3‐month period in 2009.…”

Section: Experiments With Batches Of Queriesmentioning

confidence: 99%

Performance evaluation of single vs. batch of queries on GPUs

Gaioso

Gil-Costa

Guardia

et al. 2019

Concurrency and Computation

View full text Add to dashboard Cite

Summary The WAND processing strategy is a dynamic pruning algorithm designed for large scale Web search engines where fast response to queries is a critical service. The WAND is used to reduce the amount of computation by scoring only documents that may become part of the top‐k document results. In this paper, we present two parallel strategies for the WAND algorithm and compare their performance on GPUs. In our first strategy (named size‐based), the posting lists are evenly partitioned among thread blocks. Our second strategy (named range‐based) partitions the posting lists according to document identifier intervals; thus, partitions may have different sizes. We also propose three threshold sharing policies, named Local, Safe‐R, and Safe‐WR, which emulate the WAND algorithm global pruning technique. We evaluated our proposals with different amounts of work, from short to extra‐large queries, using single query processing and batch of queries. Results show that the size‐based strategy reports the highest speedups but at the cost of low quality of results. The range‐based algorithm retrievals the exact top‐k documents and maintains a good speedup. Moreover, both strategies are capable of scaling as the amount of work is increased. In addition, there is no significant difference in the performance of the three threshold sharing policies.

show abstract

“…Works dedicated to simulate the interaction among queries, click log, and preferences of users have been built [6,10]. In addition, today with the exponential growth of the Web, simulation provides interesting approximations of performance on the Web [2,13].…”

Section: Simulation In Ir Evaluationmentioning

confidence: 99%

Evaluating the Interest of Revamping Past Search Results

Gutiérrez-Soto

Hubert

2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

OATAO is an open access repository that collects the work of Toulouse researchers and makes it freely available over the web where possible. Abstract. In this paper we present two contributions: a method to construct simulated document collections suitable for information retrieval evaluation as well as an approach of information retrieval using past queries and based on result combination. Exponential and Zipf distribution as well as Bradford's law are applied to construct simulated document collections suitable for information retrieval evaluation. Experiments comparing a traditional retrieval approach with our approach based on past queries using past queries show encouraging improvements using our approach.

show abstract

Sync/Async parallel search for the efficient design and construction of web search engines

Cited by 21 publications

References 12 publications

Distributed text search using suffix arrays

Distributed text search using suffix arrays

Performance evaluation of single vs. batch of queries on GPUs

Evaluating the Interest of Revamping Past Search Results

Contact Info

Product

Resources

About