Fast document-at-a-time query processing using two-tier indexes

Abstract

In this paper we present two new algorithms designed to reduce the overall time required to process top-k queries. These algorithms are based on the document-at-a-time approach and modify the best baseline we found in the literature, Blockmax WAND (BMW), to take advantage of a two-tiered index, in which the first tier is a small index containing only the higher impact entries of each inverted list. This small index is used to pre-process the query before accessing a larger index in the second tier, resulting in considerable speeding up the whole process. The first algorithm we propose, named BMW-CS, achieves higher performance, but may result in small changes in the top results provided in the final ranking. The second algorithm, named BMW-t, preserves the top results and, while slower than BMW-CS, it is faster than BMW. In our experiments, BMW-CS was more than 40 times faster than BMW when computing top 10 results, and, while it does not guarantee preserving the top results, it preserved all ranking results evaluated at this level.

Fast top-k preserving query processing using two-tier indexes

“…We presented experiments in our previous article ( Rossi et al, 2013 ) showing that BMW-CS could be several times faster than BMW. However, the weighing function adopted in our previous article was different from the standard adopted in the literature.…”

“…The loading times are not taken into account. All of these setup options were chosen for being similar to those adopted in previous studies ( Ding & Suel, 2011;Rossi et al, 2013;Strohman & Croft, 2007 ), including the baselines we adopt here. We ran the experiments in a 24-core Intel(R) Xeon(R), with X5680 Processor, 3.33 GHz and 64GB of primary memory.…”

“…We implemented the methods BMW and its multi-tier version, MBMW ( Ding & Suel, 2011 ), and BMW-CS ( Rossi et al, 2013 ) as baselines. To avoid a bias due to optimization of any version, all the codes were developed using the same basic BMW code, which was provided by one of the authors of BMW and MBMW Ding and Suel (2011) .…”

“…The combination of multiple sources of evidence is usually expensive, and thus it is not applied to the full set of documents that suffice the query being processed ( Carvalho, Rossi, de Moura, Fernandes, & Silva, 2012;Rossi, de Moura, Carvalho, & Silva, 2013 ). Usually, only a small subset of documents selected from an initial process of ranking is considered in a second, more sophisticated ranking phase.…”

“…This method is an improved and extended version of BMW-CS, Block Max WAND with Candidate Selection , which was also previously proposed by us ( Rossi et al, 2013 ) .…”