Using Conjunctions for Faster Disjunctive Top-k Queries

Researchers have had much recent success with ranking models based on so-called learned sparse representations generated by transformers. One crucial advantage of this approach is that such models can exploit inverted indexes for top- k retrieval, thereby leveraging decades of work on efficient query evaluation. Yet, there remain many open questions about how these learned representations fit within the existing literature, which our work aims to tackle using four representative learned sparse models. We find that impact weights generated by transformers appear to greatly reduce opportunities for skipping and early exiting optimizations in well-studied document-at-a-time ( DaaT ) approaches. Similarly, “off-the-shelf” application of score-at-a-time ( SaaT ) processing exhibits a mismatch between these weights and assumptions behind accumulator management strategies. Building on these observations, we present solutions to address deficiencies with both DaaT and SaaT approaches, yielding substantial speedups in query evaluation. Our detailed empirical analysis demonstrates that both methods lie on the effectiveness–efficiency Pareto frontier, indicating that the optimal choice for deployment depends on operational constraints.

show abstract

Using Conjunctions for Faster Disjunctive Top-k Queries

Cited by 3 publications

References 29 publications

Improved Learned Sparse Retrieval with Corpus-Specific Vocabularies

Improved Learned Sparse Retrieval with Corpus-Specific Vocabularies

A Sketch-based Index for Correlated Dataset Search

Efficient Document-at-a-Time and Score-at-a-Time Query Evaluation for Learned Sparse Representations

Contact Info

Product

Resources

About