New caching techniques for web search engines

Gil-Costa, Verónica; Gómez-Pantoja, Carlos

doi:10.1145/1851476.1851502

Cited by 33 publications

(27 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, in their study, they do not compare the two types of result caches. In a more recent study [Marin et al 2010], a docID cache, referred to as the top k cache, is investigated in a distributed query processing framework, also involving an HTML result cache and a posting list cache. Similarly, in a previous work, we propose a five-level static cache architecture that contains separate caches for HTML result pages, docID results, posting lists, intersections of posting lists, and documents.…”

Section: Search Results Cachingmentioning

confidence: 99%

Second Chance

et al. 2013

View full text Add to dashboard Cite

Web search engines are known to cache the results of previously issued queries. The stored results typically contain the document summaries and some data that is used to construct the final search result page returned to the user. An alternative strategy is to store in the cache only the result document IDs, which take much less space, allowing results of more queries to be cached. These two strategies lead to an interesting trade-off between the hit rate and the average query response latency. In this work, in order to exploit this trade-off, we propose a hybrid result caching strategy where a dynamic result cache is split into two sections: an HTML cache and a docID cache. Moreover, using a realistic cost model, we evaluate the performance of different result prefetching strategies for the proposed hybrid cache and the baseline HTML-only cache. Finally, we propose a machine learning approach to predict singleton queries, which occur only once in the query stream. We show that when the proposed hybrid result caching strategy is coupled with the singleton query predictor, the hit rate is further improved.

show abstract

Section: Search Results Cachingmentioning

confidence: 99%

Second Chance

et al. 2013

View full text Add to dashboard Cite

show abstract

“…• The architectures with the maximum number of machines were implemented by simulation: 2048 machines in [71] and 1024 machines in [23] and [26].…”

Section: Chronological Analysismentioning

confidence: 99%

“…• Also in case of big companies, they use to build their systems upon simulation platforms, such as in [71] and [6]. Real 120 120 [71] 2010 Simulation 2048 [6] 2012 Simulation 512…”

Section: Chronological Analysismentioning

confidence: 99%

Query Scheduling Techniques and Power-Latency Trade-off Model for Large-Scale Search Engines

Freire

2015

SIGIR Forum

View full text Add to dashboard Cite

AcknowledgmentsThis path would not have been possible without the help of my supervisor, FidelCacheda, who gave me the opportunity of diving again into the Information Retrieval field, after an unforgettable start in the Information Retrieval Lab. Thank you for being such a great motivating person, for guiding and advising me in the best way and, above all, for always letting me choose the next step.All my gratitude to all the colleagues that have shared with me great moments in the Telematic Engineering Lab. Thank you for making our office a warm place to spend lots of hours and for building such a great friendship even out of our blind walls.I would like to acknowledge Iadh Ounis and the Information Retrieval Group from the University of Glasgow. They gave me the opportunity of staying three months in 2012 in one of the leading groups in IR, where I put in touch with an impeccable way of work. My special gratitude goes to Craig Macdonald: thank you for becoming my best teacher and reference during all my PhD. I can't ever forget some of your encouraging words that became my motto: Don't think "is this sufficient?" but "how can we do better". All this work would not have being possible without your valuable help. A special gratitude also for Silvia Lorenzo Freire, for sharing with me her huge knowledge and offering me her useful help. Vorrei ringraziare in modo particolare a tutti i membri del HighI can not forget Roi Blanco, who was discreetly present at every milestone of my career. Thank you also for opening me the next opportunity of learning, I will make the most of it.A great deal of gratitude is due to all the people that have walked with me during this period. Your encouraging and warm-hearted words were the best incentive to finish this work. Those who kept me out of this thesis, sharing awesome moments and messages, have also done a good job. AbstractWeb search engines have to deal with a rapid increase of information, demanded by high incoming query traffic. This situation has driven companies to build geographically distributed data centres housing thousands of computers, consuming enormous amounts of electricity and requiring a huge infrastructure around. At this scale, even minor efficiency improvements result in large financial savings.This thesis represents a novel contribution to query scheduling and power consumption state-of-the-art, by assisting large-scale data centres to build more efficient search engines.On the one hand, this thesis proposes new scheduling techniques to decrease the response time of queries, by estimating the server that will be idle soonest.On the other hand, this thesis defines a simple mathematical model that establishes a threshold between the power and latency of a search engine. Using historical and current data, the model estimates the incoming query traffic and automatically increases/decreases the necessary number of active machines in the system. We achieve high energy savings during the whole day, without degrading the latency.Our experiments have attested th...

show abstract

“…Recently, the research community has proposed some approaches for improving the energy efficiency of search engines data centers. These can involve workload consolidation among multiple query servers [6], caching mechanisms [8,12,14] or query routing between different data centers [10,18]. These works focus on the behavior of either the whole distributed infrastructure or the single data-centre of a Web search engine.…”

Section: Introductionmentioning

confidence: 99%

Load-sensitive CPU Power Management for Web Search Engines

Catena

Macdonald

Tonellotto

2015

Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

Web search engine companies require power-hungry data centers with thousands of servers to efficiently perform searches on a large scale. This permits the search engines to serve high arrival rates of user queries with low latency, but poses economical and environmental concerns due to the power consumption of the servers. Existing power saving techniques sacrifice the raw performance of a server for reduced power absorption, by scaling the frequency of the server's CPU according to its utilization. For instance, current Linux kernels include frequency governors i.e., mechanisms designed to dynamically throttle the CPU operational frequency. However, such general-domain techniques work at the operating system level and have no knowledge about the querying operations of the server. In this work, we propose to delegate CPU power management to search engine-specific governors. These can leverage knowledge coming from the querying operations, such as the query server utilization and load. By exploiting such additional knowledge, we can appropriately throttle the CPU frequency thereby reducing the query server power consumption. Experiments are conducted upon the TREC ClueWeb09 corpus and the query stream from the MSN 2006 query log. Results show that we can reduce up to ∼24% a server power consumption, with only limited drawbacks in effectiveness w.r.t. a system running at maximum CPU frequency to promote query processing quality.

show abstract

New caching techniques for web search engines

Cited by 33 publications

References 21 publications

Second Chance

Second Chance

Query Scheduling Techniques and Power-Latency Trade-off Model for Large-Scale Search Engines

Load-sensitive CPU Power Management for Web Search Engines

Contact Info

Product

Resources

About