Versioning a full-text information retrieval system

Anick, Peter; Flynn, Rex A.

doi:10.1145/133160.133183

Cited by 27 publications

(34 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One of the earliest known efforts in this direction is by Anick and Flynn [8] who developed a framework for versioning the complete index for historical queries. Recently, Alonso et al [6] give an overview of relevant research directions.…”

Section: Related Workmentioning

confidence: 99%

Temporal index sharding for space-time efficiency in archive search

Anand

Bedathur

Berberich

et al. 2011

Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

Time-travel queries that couple temporal constraints with keyword queries are useful in searching large-scale archives of time-evolving content such as the web archives or wikis. Typical approaches for efficient evaluation of these queries involve slicing either the entire collection [20] or individual index lists [10] along the time-axis. Both these methods are not satisfactory since they sacrifice compactness of index for processing efficiency making them either too big or, otherwise, too slow.We present a novel index organization scheme that shards each index list with almost zero increase in index size but still minimizes the cost of reading index entries during query processing. Based on the optimal sharding thus obtained, we develop a practically efficient sharding that takes into account the different costs of random and sequential accesses. Our algorithm merges shards from the optimal solution to allow for a few extra sequential accesses while gaining significantly by reducing the number of random accesses. We empirically establish the effectiveness of our sharding scheme with experiments over the revision history of the English Wikipedia between 2001-2005 (≈ 700 GB) and an archive of U.K. governmental web sites (≈ 400 GB). Our results demonstrate the feasibility of faster time-travel query processing with no space overhead.

show abstract

Section: Related Workmentioning

confidence: 99%

Temporal index sharding for space-time efficiency in archive search

Anand

Bedathur

Berberich

et al. 2011

Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

show abstract

“…Anick and Flynn [3], while pioneering this research, describe a help-desk system that supports historical queries. Access costs are optimized for accesses to the most recent versions and increase as one moves farther into the past.…”

Section: Related Workmentioning

confidence: 99%

A time machine for text search

Berberich

Bedathur

Neumann

et al. 2007

Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

Text search over temporally versioned document collections such as web archives has received little attention as a research problem. As a consequence, there is no scalable and principled solution to search such a collection as of a specified time t. In this work, we address this shortcoming and propose an efficient solution for time-travel text search by extending the inverted file index to make it ready for temporal search. We introduce approximate temporal coalescing as a tunable method to reduce the index size without significantly affecting the quality of results. In order to further improve the performance of time-travel queries, we introduce two principled techniques to trade off index size for its performance. These techniques can be formulated as optimization problems that can be solved to near-optimality. Finally, our approach is evaluated in a comprehensive series of experiments on two large-scale real-world datasets. Results unequivocally show that our methods make it possible to build an efficient "time machine" scalable to large versioned text collections.

show abstract

“…We are only aware of three indexing approaches that directly address the issue of temporal text-indexing: The Anick and Flynn approach [1], V2 [10], and ITTX [9,12]. In the proposal of Anick and Flynn, the timestamp as version identifier.…”

Section: Related Workmentioning

confidence: 99%