Faster Learned Sparse Retrieval with Guided Traversal

Mallia, Antonio; Mackenzie, Joel; Suel, Torsten; Tonellotto, Nicola

doi:10.1145/3477495.3531774

Cited by 21 publications

(19 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…LSR is compatible with many techniques from sparse retrieval, such as inverted indexing and accompanying query processing algorithms. However, differences in LSR weights can mean that existing query processing optimizations become much less helpful, motivating new optimizations [21,22,24].…”

Section: Learned Sparse Retrievalmentioning

confidence: 99%

A Unified Framework for Learned Sparse Retrieval

Nguyen

MacAvaney

Yates

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Learned sparse retrieval (LSR) is a family of first-stage retrieval methods that are trained to generate sparse lexical representations of queries and documents for use with an inverted index. Many LSR methods have been recently introduced, with Splade models achieving state-of-the-art performance on MSMarco. Despite similarities in their model architectures, many LSR methods show substantial differences in effectiveness and efficiency. Differences in the experimental setups and configurations used make it difficult to compare the methods and derive insights. In this work, we analyze existing LSR methods and identify key components to establish an LSR framework that unifies all LSR methods under the same perspective. We then reproduce all prominent methods using a common codebase and re-train them in the same environment, which allows us to quantify how components of the framework affect effectiveness and efficiency. We find that (1) including document term weighting is most important for a method's effectiveness, (2) including query weighting has a small positive impact, and (3) document expansion and query expansion have a cancellation effect. As a result, we show how removing query expansion from a state-of-the-art model can reduce latency significantly while maintaining effectiveness on MSMarco and TripClick benchmarks. Our code is publicly available. 3

show abstract

Section: Learned Sparse Retrievalmentioning

confidence: 99%

A Unified Framework for Learned Sparse Retrieval

Nguyen

MacAvaney

Yates

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Similarly, uniCOIL [24] extends the work of COIL [15] for contextualized term weights. Document retrieval with term weights learned from a transformer has been found slow in [29,31]. Mallia et al [31] state that the MaxScore retrieval algorithm does not efficiently exploit the DeepImpact scores.…”

Section: Background and Related Workmentioning

confidence: 99%

“…Document retrieval with term weights learned from a transformer has been found slow in [29,31]. Mallia et al [31] state that the MaxScore retrieval algorithm does not efficiently exploit the DeepImpact scores. Mackenzie et al [29] view that the learned sparse term weights are "wacky" as they affect document skipping during retrieval thus they advocate ranking approximation with score-at-a-time traversal.…”

Section: Background and Related Workmentioning

confidence: 99%

“…Mallia et al [31] propose to use BM25 to guide traversal, called GT, for fast learned sparse retrieval because the distribution of learned weights results in less pruning opportunities and they conducted an evaluation with retrieval model DeepImpact [30]. One variation they propose is to compute the final rank scoring as a linear combination of the learned weights and BM25 weights, denoted as GTI.…”

Section: Introductionmentioning

confidence: 99%

“…GT is a special case of GTI and this paper treats GTI as the main baseline. Since the BM25 weight for a document term pair may not exist in a learned sparse index, zero filling is used in Mallia et al [31] to align the BM25 and learned weight models. During our evaluation using GT for SPLADE v2 and its revision SPLADE++ [12,13], we find that as retrieval depth 𝑘 decreases, BM25 driven skipping becomes too aggressive in dropping documents desired by top 𝑘 ranking based on learned term weights, which can cause a significant relevance degradation.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Optimizing Guided Traversal for Fast Learned Sparse Retrieval

Qiao

Yang

Lin

et al. 2023

Proceedings of the ACM Web Conference 2023

View full text Add to dashboard Cite

Recent studies show that BM25-driven dynamic index skipping can greatly accelerate MaxScore-based document retrieval based on the learned sparse representation derived by DeepImpact. This paper investigates the effectiveness of such a traversal guidance strategy during top 𝑘 retrieval when using other models such as SPLADE and uniCOIL, and finds that unconstrained BM25-driven skipping could have a visible relevance degradation when the BM25 model is not well aligned with a learned weight model or when retrieval depth 𝑘 is small. This paper generalizes the previous work and optimizes the BM25 guided index traversal with a two-level pruning control scheme and model alignment for fast retrieval using a sparse representation. Although there can be a cost of increased latency, the proposed scheme is much faster than the original MaxScore method without BM25 guidance while retaining the relevance effectiveness. This paper analyzes the competitiveness of this two-level pruning scheme, and evaluates its tradeoff in ranking relevance and time efficiency when searching several test datasets.

show abstract