ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction

Abstract

Neural information retrieval (IR) has greatly advanced search and other knowledgeintensive language tasks. While many neural IR methods encode queries and documents into single-vector representations, late interaction models produce multi-vector representations at the granularity of each token and decompose relevance modeling into scalable token-level computations. This decomposition has been shown to make late interaction more effective, but it inflates the space footprint of these models by an order of magnitude. In this work, we introduce ColBERTv2, a retriever that couples an aggressive residual compression mechanism with a denoised supervision strategy to simultaneously improve the quality and space footprint of late interaction. We evaluate ColBERTv2 across a wide range of benchmarks, establishing state-of-the-art quality within and outside the training domain while reducing the space footprint of late interaction models by 6-10×.

Fast and Light-Weight Answer Text Retrieval in Dialogue Systems

“…Dense passage retrieval Khattab and Zaharia, 2020;Luan et al, 2021;Santhanam et al, 2021) has gained a lot of attention lately with applications extending beyond retrieval tasks into areas including open-domain question answering, language model pre-training, fact checking, dialogue generation (e.g., RAG , REALM (Guu et al, 2020), MultiDPR , KILT , Con-vDR , RocketQA (Qu et al, 2021)). In dense passage retrieval, the query q and each passage p are separately encoded into dense vectors, and relevance is modeled via similarity functions such as dot-product.…”

“…Dense passage retrieval methods Khattab and Zaharia, 2020;Santhanam et al, 2021;Luan et al, 2021;Humeau et al, 2020;MacAvaney et al, 2020; have gained a lot of attention lately and achieved state of the art results on various retrieval and ranking datasets. Dense retrievers are efficient compared to other neural methods such as transformer-based cross-encoder models: passages are encoded and indexed offline, at inference time only the query needs to be encoded once; also they leverage ANN (approximate nearest neighbor) algorithms to efficiently search for relevant dense vectors.…”

“…In late interaction models such as Col-BERT (Khattab and Zaharia, 2020;Santhanam et al, 2021), the query and passages are separately encoded to obtain query token vectors and passage token vectors. These models adopt token-decomposed scoring, e.g.…”

“…Dimension reduction (Khattab and Zaharia, 2020;Santhanam et al, 2021) showed that a ColBERT model with quantized and reduced-dimension vectors could perform comparably to the standard model on big retrieval/ranking benchmarks while greatly reducing the space requirement for saving the final representations. For our use case on the small retrieval datasets, we explored using smaller dimensions for the vector representations in Col-BERT.…”

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Track

“…Dense passage retrieval Luan et al, 2021;Santhanam et al, 2021) has gained a lot of attention lately with applications extending beyond retrieval tasks into areas including open-domain question answering, language model pre-training, fact checking, dialogue generation (e.g., RAG , REALM (Guu et al, 2020), MultiDPR , KILT , Con-vDR , RocketQA (Qu et al, 2021)). In dense passage retrieval, the query q and each passage p are separately encoded into dense vectors, and relevance is modeled via similarity functions such as dot-product.…”