Temporal index sharding for space-time efficiency in archive search

Anand, Avishek; Bedathur, Srikanta; Berberich, Klaus; Schenkel, Ralf

doi:10.1145/2009916.2009991

Cited by 16 publications

(21 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The approach proposed in [6] is to shard (or horizontally partition) each term list along the document identifiers instead of time. Entries in a term list are thus distributed over disjoint sub-lists called shards, and entries in a shard are ordered according to their start times t s .…”

Section: Previous Methodsmentioning

confidence: 99%

“…An optimal greedy algorithm for creating this partitioning is given in [6]; an example of temporal sharding for the term list from Fig. 1, is shown in Fig.…”

Section: Previous Methodsmentioning

confidence: 99%

“…The data model for versioned document collections was formally introduced in [10], and used by later works [9,5,6]. Let D be a set of n documents d 1 [7], language models [13] and Okapi BM25 [14].…”

Section: Definitionsmentioning

confidence: 99%

See 2 more Smart Citations

A Comparison of Top-k Temporal Keyword Querying over Versioned Text Collections

Huo

Tsotras

2012

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. As the web evolves over time, the amount of versioned text collections increases rapidly. Most web search engines will answer a query by ranking all known documents at the (current) time the query is posed. There are applications however (for example customer behavior analysis, crime investigation, etc.) that would need to efficiently query these sources as of some past time, that is, retrieve the results as if the user was posing the query in a past time instant, thus accessing data known as of that time. Ranking and searching over versioned documents considers not only keyword constraints but also the time dimension, most commonly, a time point or time range of interest. In this paper, we deal with top-k query evaluations with both keyword and temporal constraints over versioned textual documents. In addition to considering previous solutions, we propose novel data organization and indexing solutions: the first one partitions data along ranking positions, while the other maintains the full ranking order through the use of a multiversion ordered list. We present an experimental comparison for both time point and time interval constraints. For time-interval constraints, different querying definitions, such as aggregation functions and consistent top-k queries are evaluated. Experimental evaluations on large real world datasets demonstrate the advantages of the newly proposed data organization and indexing approaches. If a text collection does not retain past documents, then a search query ranks only the documents as of the most current time. If the collection contains versioned documents, a search typically considers each version of a document as a separate document and the ranking is taken over all documents independently to the document's version (creation time). There are applications however, where this approach is not Introduction

show abstract

Section: Previous Methodsmentioning

confidence: 99%

“…An optimal greedy algorithm for creating this partitioning is given in [6]; an example of temporal sharding for the term list from Fig. 1, is shown in Fig.…”

Section: Previous Methodsmentioning

confidence: 99%

See 1 more Smart Citation

A Comparison of Top-k Temporal Keyword Querying over Versioned Text Collections

Huo

Tsotras

2012

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…This important subfield of IR has the goal to improve search effectiveness by exploiting temporal information in documents and queries [11,12]. The temporal dimension leads to new challenges in query understanding [13], retrieval models [14,15] as well as temporal indexing [16,17]. However, most temporal indexing approaches treat documents as static texts with a certain validity, which does not account for the dynamics in Web archives as described above.…”

Section: Web Archive Searchmentioning

confidence: 99%

Accessing web archives from different perspectives with potential synergies

Holzmann¹,

Risse²

2017

Researchers, Practitioners and Their Use of the Archived Web

View full text Add to dashboard Cite

Abstract. Web archives constitute a valuable source for researchers from many disciplines. However, their sheer size, the typically broad scope and their temporal dimension make them difficult to work with. We have identified three approaches to access and explore Web archives from different perspectives: user-, data-and graph-centric. In this paper we present related works on these three views as well as discuss their relations and potential synergies. Finally, we propose a generic analysis schema that outlines a systematic way to study Web archives by approaching them from different zoom levels corresponding to the three presented views.

show abstract

“…Users can thus combine a keyword query (e.g., financial crisis) with a time interval (e.g., [2006,2008]) to retrieve all document versions from the archive that are considered relevant to the given keywords and existed during the given time interval. Different index structures [3,4,7] have been proposed to efficiently support time-travel text search, incurring controllable overhead either in terms of index size or response time when compared to standard text search.…”

Section: Introductionmentioning

confidence: 99%

Index maintenance for time-travel text search

Anand

Bedathur

Berberich

et al. 2012

Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval

Self Cite

View full text Add to dashboard Cite

Time-travel text search enriches standard text search by temporal predicates, so that users of web archives can easily retrieve document versions that are considered relevant to a given keyword query and existed during a given time interval. Different index structures have been proposed to efficiently support time-travel text search. None of them, however, can easily be updated as the Web evolves and new document versions are added to the web archive.In this work, we describe a novel index structure that efficiently supports time-travel text search and can be maintained incrementally as new document versions are added to the web archive. Our solution uses a sharded index organization, bounds the number of spuriously read index entries per shard, and can be maintained using small in-memory buffers and append-only operations. We present experiments on two large-scale real-world datasets demonstrating that maintaining our novel index structure is an order of magnitude more efficient than periodically rebuilding one of the existing index structures, while query-processing performance is not adversely affected.

show abstract

Temporal index sharding for space-time efficiency in archive search

Cited by 16 publications

References 18 publications

A Comparison of Top-k Temporal Keyword Querying over Versioned Text Collections

A Comparison of Top-k Temporal Keyword Querying over Versioned Text Collections

Accessing web archives from different perspectives with potential synergies

Index maintenance for time-travel text search

Contact Info

Product

Resources

About