An Efficiency Study for SPLADE Models

Abstract

Latency and efficiency issues are often overlooked when evaluating IR models based on Pretrained Language Models (PLMs) in reason of multiple hardware and software testing scenarios. Nevertheless, efficiency is an important part of such systems and should not be overlooked.In this paper, we focus on improving the efficiency of the SPLADE model since it has achieved state-of-the-art zero-shot performance and competitive results on TREC collections. SPLADE efficiency can be controlled via a regularization factor, but solely controlling this regularization has been shown to not be efficient enough. In order to reduce the latency gap between SPLADE and traditional retrieval systems, we propose several techniques including L1 regularization for queries, a separation of document/query encoders, a FLOPSregularized middle-training, and the use of faster query encoders. Our benchmark demonstrates that we can drastically improve the efficiency of these models while increasing the performance metrics on in-domain data. To our knowledge, we propose the first neural models that, under the same computing constraints, achieve similar latency (less than 4ms difference) as traditional BM25, while having similar performance (less than 10% MRR@10 reduction) as the stateof-the-art single-stage neural rankers on in-domain data.

A Unified Framework for Learned Sparse Retrieval

“…We confirmed this by replacing the shared encoder with two separate ones (distilSplade sep ), which reduced latency from 122.5 ms to 50.2 ms, a 59% decrease. This benefit of separate encoders was also reported in [14], and our results further support its substantial impact.…”

“…In practice, distilSplade qM LP could be viewed as a more efficient drop-in replacement for the full model. This use of qM LP is complementary to other changes (e.g., using a smaller encoder as in [14]) to improve the efficiency of LSR.…”

Optimizing Guided Traversal for Fast Learned Sparse Retrieval

“…Efficient SPLADE model. Table 5 shows the application of 2GTI in a recently published efficient SPLADE model [20] which has made several improvements in retrieval speed. We have used the released checkpoint of this efficient model called BT-SPLADE-L, which has a weaker MRR@10 score, but significantly faster than our trained SPLADE baseline reported in Table 2.…”

Extending English IR methods to multi-lingual IR

“…We follow the strategy we used on our latest TREC notebooks, in that we strive for making this more streamlined than a normal research paper would be. We will now present a list of the papers that better introduce and detail the models we used here and refer the reader to check them for a better explanation than those we have here, that are mainly dedicated to how to apply it to MIRACL and not to the methods themselves: i) Training non English SPLADE models [11], ii) The SPLADE model [5,10], iii) The Contriever model and its pretraining [8], iv) The RankT5 reranker [16], v) MonoT5 [13], vi) The LCE loss [6], vii) ColBERT [9], and viii) For our ensembling we use Ranx [1] and their min-max normalized sum ensembling.…”

“…This was mostly due to desperation when we saw everyone overtaking us on the dev set. These rerankers were then tested on known languages and they did not improve the results 10. https://huggingface.co/google/byt5-xl 11 https://huggingface.co/microsoft/mdeberta-v3-base…”