Efficient Neural Ranking using Forward Indexes

Leonhardt, Jurek; Rudra, Koustav; Khosla, Megha; Anand, Abhijit; Anand, Avishek

doi:10.1145/3485447.3511955

Cited by 9 publications

(8 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Re-ranking. Both LADR variants build upon the high effectiveness of the re-ranking technique [20,41]. Using a lexical model for reranking alone is a major drawback, however: it inherently limits the retrievable documents to those with lexical matches.…”

Section: Comparison With Existing Methodsmentioning

confidence: 99%

“…Most notably, they suffer in terms of recall, which is meant to be one of the key benefits of dense retrieval. Remarkably, simply re-ranking an initial pool of lexical results (such as BM25) remains highly competitive [20,41]. We posit that this is due to two reasons.…”

Section: R@1k Ladrmentioning

confidence: 95%

“…Efficient lexical methods are used at the first stage to retrieve an initial pool of documents based on the re-ranking budget. Effective dense retrieval methods are used at the second stage to re-rank the obtained results at the first stage [20,25,41]. Re-ranking pipelines perform very well but are limited by recall.…”

Section: Re-ranking Pipeline To Improve Efficiencymentioning

confidence: 99%

See 2 more Smart Citations

Lexically-Accelerated Dense Retrieval

Kulkarni

MacAvaney

Goharian

2023

Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

Retrieval approaches that score documents based on learned dense vectors (i.e., dense retrieval) rather than lexical signals (i.e., conventional retrieval) are increasingly popular. Their ability to identify related documents that do not necessarily contain the same terms as those appearing in the user's query (thereby improving recall) is one of their key advantages. However, to actually achieve these gains, dense retrieval approaches typically require an exhaustive search over the document collection, making them considerably more expensive at query-time than conventional lexical approaches. Several techniques aim to reduce this computational overhead by approximating the results of a full dense retriever. Although these approaches reasonably approximate the top results, they suffer in terms of recall -one of the key advantages of dense retrieval. We introduce 'LADR' (Lexically-Accelerated Dense Retrieval), a simpleyet-effective approach that improves the efficiency of existing dense retrieval models without compromising on retrieval effectiveness. LADR uses lexical retrieval techniques to seed a dense retrieval exploration that uses a document proximity graph. Through extensive experiments, we find that LADR establishes a new dense retrieval effectiveness-efficiency Pareto frontier among approximate 𝑘 nearest neighbor techniques. When tuned to take around 8ms per query in retrieval latency on our hardware, LADR consistently achieves both precision and recall that are on par with an exhaustive search on standard benchmarks. Importantly, LADR accomplishes this using only a single CPU -no hardware accelerators such as GPUs -which reduces the deployment cost of dense retrieval systems. CCS CONCEPTS• Information systems → Retrieval models and ranking.

show abstract

Section: Comparison With Existing Methodsmentioning

confidence: 99%

Section: R@1k Ladrmentioning

confidence: 95%

Section: Re-ranking Pipeline To Improve Efficiencymentioning

confidence: 99%

See 1 more Smart Citation

Lexically-Accelerated Dense Retrieval

Kulkarni

MacAvaney

Goharian

2023

Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

show abstract

“…TILDEv2 [34] uses exact contextualized term matching to reduce the memory requirements. Leonhardt et al [20] propose vector forward indexes that allow for efficient interpolationbased re-ranking using dual-encoders.…”

Section: Neural Retrieval and Rankingmentioning

confidence: 99%

“…The relevance of a query-document pair is then computed as the dot product of the query and document representation vectors. This is referred to as two-tower, bi-encoder or dual-encoder architecture and has been used for retrieval [14,16,22] and re-ranking [15,20,35]. Typically, the query and document encoder either (1) are architecturally identical and initialized using the same pre-trained model or (2) even share their weights in a Siamese fashion.…”

Section: Introductionmentioning

confidence: 99%

Distribution-Aligned Fine-Tuning for Efficient Neural Retrieval

Leonhardt¹,

Jahnke²,

Anand³

2022

Preprint

View full text Add to dashboard Cite

Dual-encoder-based neural retrieval models achieve appreciable performance and complement traditional lexical retrievers well due to their semantic matching capabilities, which makes them a common choice for hybrid IR systems. However, these models exhibit a performance bottleneck in the online query encoding step, as the corresponding query encoders are usually large and complex Transformer models.In this paper we investigate heterogeneous dual-encoder models, where the two encoders are separate models that do not share parameters or initializations. We empirically show that heterogeneous dual-encoders are susceptible to collapsing representations, causing them to output constant trivial representations when they are fine-tuned using a standard contrastive loss due to a distribution mismatch. We propose DAFT, a simple two-stage fine-tuning approach that aligns the two encoders in order to prevent them from collapsing. We further demonstrate how DAFT can be used to train efficient heterogeneous dual-encoder models using lightweight query encoders.

show abstract