Two-Dimensional Distributed Inverted Files

Feuerstein, Esteban; Mizrahi, Michel J.; Gil-Costa, Verónica; Baeza-Yates, Ricardo

doi:10.1007/978-3-642-03784-9_20

Cited by 6 publications

(4 citation statements)

References 15 publications

(11 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Replication is used as a mean to increase query throughput and to support failures. The 3D index efficiently handles replication and outperforms the 2D index which in turn outperforms well-known distributed indexes [5]. The number of rows, columns and replicas can be chosen so as to optimize the overall performance.…”

Section: Discussionmentioning

confidence: 99%

“…The 2D index that combines document-and term-partitioning was originally proposed in [5]. Other variants of these basic schemes have been proposed in [11,13].…”

Section: Related Workmentioning

confidence: 99%

“…In our previous work [5], a 2-dimensional (2D) index was introduced. The 2D index combines document-and term-partitioning to get the "best of two worlds", i.e.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

3D Inverted Index with Cache Sharing for Web Search Engines

Feuerstein¹,

Gil-Costa

Tolosa

et al. 2012

Euro-Par 2012 Parallel Processing

Self Cite

View full text Add to dashboard Cite

Web search engines achieve efficient performance by partitioning and replicating the indexing data structure used to support query processing. Current practice simply partitions and replicates the text collection on the set of cluster processors and then constructs in each processor an index data structure. This paper proposes a different approach by constructing an index data structure that properly considers the fact that data is partitioned and replicated. This leads to a so-called 3D indexing strategy that outperforms current approaches. Performance is further boosted by introducing an application caching scheme devised to hold most frequently issued queries.

show abstract

Section: Discussionmentioning

confidence: 99%

“…The 2D index that combines document-and term-partitioning was originally proposed in [5]. Other variants of these basic schemes have been proposed in [11,13].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

3D Inverted Index with Cache Sharing for Web Search Engines

Feuerstein¹,

Gil-Costa

Tolosa

et al. 2012

Euro-Par 2012 Parallel Processing

Self Cite

View full text Add to dashboard Cite

show abstract

“…shows the concept of providing a one-integrated-memory view of collective memories by using two-dimensional indexing over n × m two-dimensional distributed memory. Here, the horizontal axis of the array indicates n index shards that are partitioned from the entire index; the vertical axis in each column of the array indicates m index fragments that are partitioned from each index shard as in Feuerstein et al (2009). In addition, each index fragment in main-memory is associated with a full (non-keyword-partitioned) index shard on disk.…”

mentioning

confidence: 99%

Correction to: Two-dimensional indexing to provide one-integrated-memory view of distributed memory for a massively-parallel search engine

et al. 2019

View full text Add to dashboard Cite

Feuerstein et al. (2009). We regret that items 1 and 3 in Section 1.4 should be corrected to read as follows: − (Item 1) We adopt the notion of two-dimensional distributed inverted files proposed by Feuerstein et al. (2009) with the purpose of optimizing the entire system cost including computation, communication, and fixed overheads of the system. Feuerstein et al. (2009) allocated index fragments in a two-dimensional array of processors through keyword (or term) partitioning (defined in Section 2.2) and document partitioning (defined in Section 2.2). In this paper, we adopt this basic scheme with a different purpose of minimizing the average query response time by simulating collective distributed (main) memory as one integrated memory. Then, we extend this scheme so that each row of the twodimensional array stores a replicated (document-partitioned only) index shard on disk for efficient processing of multiple-keyword queries. We call this extended scheme and associated algorithms two-dimensional indexing.

show abstract