Allocating inverted index into flash memory for search engines

Huang, Bojun; Xia, Zenglin

doi:10.1145/1963192.1963224

Cited by 9 publications

(4 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The issue of allocating indexes on SSD was recently studied in 2011 [75]. Last year, the issue of maintaining inverted index resided on SSD was also discussed [76].…”

Section: Related Studiesmentioning

confidence: 99%

The impact of solid state drive on search engine cache management

Wang

Yiu

et al. 2013

Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

Caching is an important optimization in search engine architectures. Existing caching techniques for search engine optimization are mostly biased towards the reduction of random accesses to disks, because random accesses are known to be much more expensive than sequential accesses in traditional magnetic hard disk drive (HDD). Recently, solid state drive (SSD) has emerged as a new kind of secondary storage medium, and some search engines like Baidu have already used SSD to completely replace HDD in their infrastructure. One notable property of SSD is that its random access latency is comparable to its sequential access latency. Therefore, the use of SSDs to replace HDDs in a search engine infrastructure may void the cache management of existing search engines. In this paper, we carry out a series of empirical experiments to study the impact of SSD on search engine cache management. The results give insights to practitioners and researchers on how to adapt the infrastructure and how to redesign the caching policies for SSDbased search engines.

show abstract

“…The issue of allocating indexes on SSD was recently studied in 2011 [75]. Last year, the issue of maintaining inverted index resided on SSD was also discussed [76].…”

Section: Related Studiesmentioning

confidence: 99%

The impact of solid state drive on search engine cache management

Wang

Yiu

et al. 2013

Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

show abstract

“…Knapsack problem has been developed and used in search applications [32] [33]. Huang and Xia [34] modeled a knapsack problem based on the bandwidth and the capacity for a search engine to allocate the inverted index in the flash memory. And they also proposed the standard greedy algorithm to approximate the optimal solution.…”

Section: Knapsack Problemmentioning

confidence: 99%

EDS: An Efficient Data Selection policy for search engine storage architectures

Dong

et al. 2017

Future Generation Computer Systems

View full text Add to dashboard Cite

“…in (5), considering N tends to infinity, namely the total number of terms in document collection tends to infinity, with 1   , we have:…”

Section: A Proposed Uniform Storage Modelmentioning

confidence: 99%

“…On-line information retrieval system is a key and challenge technology to manage and search information efficiently [1,2]. Inverted index [3], as one of efficient index files for on-line information retrieval, has been comprehensively studied in recent years [4,5]. Although compression schemes [6,7] can greatly reduce disk access time, the compressed index for each query term must be completely decompressed, which will degrade query performance to some extent, especially for huge amount of text.…”

Section: Introductionmentioning

confidence: 99%

Uniform Storage Model-based Update Scheme of On-line Information Retrieval System

Liu¹,

Peng²,

He³

et al. 2013

JNW

View full text Add to dashboard Cite

In order to improve the retrieval performance of on-line information retrieval systems, an efficient index update scheme is proposed in this paper, which can provide better skipping function and further enhance both space and time efficiencies without inserting any additional auxiliary information. A uniform storage model (USM) is proposed to manage both short and long postings lists based on link. A USM-based update scheme also is proposed to distinguish long and short posting lists, which merges short lists with immediately merge, and merges long lists with improved Y-limited contiguous multiple merge scheme, which balances the trade-off of the time and space efficiencies effectively. The proposed update scheme not only considers both index level and inverted list level update, but also effectively improves time and space efficiencies of index update. Detailed experimental results and comparison with existed schemes show that the proposed scheme greatly averagely reduces space cost, conjunctive Boolean query time, and the cost of on-line index constructio

show abstract

Allocating inverted index into flash memory for search engines

Cited by 9 publications

References 3 publications

The impact of solid state drive on search engine cache management

The impact of solid state drive on search engine cache management

EDS: An Efficient Data Selection policy for search engine storage architectures

Uniform Storage Model-based Update Scheme of On-line Information Retrieval System

Contact Info

Product

Resources

About