Optimization for dynamic inverted index maintenance

Cutting, Douglass R.; Pedersen, Jan O.

doi:10.1145/96749.98245

Cited by 132 publications

(58 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…When the inmemory index exceeds its limit, it is merged with on-disk index. The simplest form of merge-based update is Immediate Merge, described by Cutting et al [6]. In this approach, only one on-disk inverted index exists.…”

Section: Merge-based Updatementioning

confidence: 99%

See 1 more Smart Citation

On-line index maintenance using horizontal partitioning

Gurajada

2009

Proceedings of the 18th ACM Conference on Information and Knowledge Management

View full text Add to dashboard Cite

In this paper, we propose a new merge-based index maintenance strategy for Information Retrieval systems. The new model is based on partitioning of the inverted index across the terms in it. We exploit the query log to partition the on-disk inverted index into two types of sub-indexes. Inverted lists of the terms contained in the queries that are frequently posed to the Information Retrieval systems are kept in one partition, called frequent-term index and the other inverted lists form another partition, called infrequentterm index. We use a lazy-merge strategy for maintaining infrequent-term sub-indexes, and an active merge strategy for maintaining frequent-term sub-indexes. The sub-indexes are also similarly split into frequent and in-frequent parts. Experimental results show that the proposed method improves both index maintenance performance and query performance compared to the existing merge-based strategies.

show abstract

Section: Merge-based Updatementioning

confidence: 99%

“…Initially [2,6,16] addressed the problem of on-line indexing. A decade after, Lester et al [14] readdressed the problem and gave a comparative study of three basic approaches: In-place, Re-Merge, and Re-Build, and showed that In-place and Remerge outperform Re-build in all cases.…”

Section: Introductionmentioning

confidence: 99%

On-line index maintenance using horizontal partitioning

Gurajada

2009

Proceedings of the 18th ACM Conference on Information and Knowledge Management

View full text Add to dashboard Cite

show abstract

“…First, web documents are more dynamic than the text document collections assumed by those techniques. Past work on incremental updates of inverted indexes [3,8,13,27,31] deals mostly with additions of new documents to a collection of static documents. Static documents are existing documents in the collection whose contents do not change over time.…”

Section: Relation With Prior Workmentioning

confidence: 99%

Efficient Update of Indexes for Dynamically Changing Web Documents

et al. 2007

View full text Add to dashboard Cite

“…The Immediate Merge strategy was first described by Cutting and Pedersen [4]. In their work, they present an index structure based on a B-tree and give a comparison of leave node caching (a form of in-place update) and index remerging.…”

Section: Merge-based Index Maintenancementioning

confidence: 99%

A Hybrid Approach to Index Maintenance in Dynamic Text Retrieval Systems

Büttcher

Clarke

2006

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. In-place and merge-based index maintenance are the two main competing strategies for on-line index construction in dynamic information retrieval systems based on inverted lists. Motivated by recent results for both strategies, we investigate possible combinations of in-place and merge-based index maintenance. We present a hybrid approach in which long posting lists are updated in-place, while short lists are updated using a merge strategy. Our experimental results show that this hybrid approach achieves better indexing performance than either method (in-place, merge-based) alone.

show abstract

Optimization for dynamic inverted index maintenance

Abstract: For free-text search over rapidly evolving corpora, dy-

Cited by 132 publications

References 4 publications

On-line index maintenance using horizontal partitioning

On-line index maintenance using horizontal partitioning

Efficient Update of Indexes for Dynamically Changing Web Documents

A Hybrid Approach to Index Maintenance in Dynamic Text Retrieval Systems

Contact Info

Product

Resources

About