UnifieR: A Unified Retriever for Large-Scale Retrieval

Abstract

Large-scale retrieval is to recall relevant documents from a huge collection given a query. It relies on representation learning to embed documents and queries into a common semantic encoding space. According to the encoding space, recent retrieval methods based on pre-trained language models (PLM) can be coarsely categorized into either dense-vector or lexicon-based paradigms. These two paradigms unveil the PLMs' representation capability in different granularities, i.e., global sequence-level compression and local word-level contexts, respectively. Inspired by their complementary global-local contextualization and distinct representing views, we propose a new learning framework, UnifieR, which unifies dense-vector and lexicon-based retrieval in one model with a dual-representing capability. Experiments on passage retrieval benchmarks verify its effectiveness in both paradigms. A uni-retrieval scheme is further presented with even better retrieval quality. We lastly evaluate the model on BEIR benchmark to verify its transferability. * Corresponding author. 2 A collection entry could be sentence, passage, document, etc., and we take document for demonstrations.Preprint. Under review.

Information Retrieval: Recent Advances and Beyond

“…[187] proposed a method called Representational Slicing which densifies sparse representations for passage retrieval. [188]) proposed a method called UnifieR which aims to improve the efficiency and effectiveness of large-scale retrieval by unifying different retrieval strategies.…”

Adaptive Hyper-parameter Learning for Deep Semantic Retrieval

“…Compared with traditional lexical-based methods, the deep semantic retrieval model has the advantages of high accuracy, low mismatch, and strong generalization. The classical deep semantic retrieval methods could be split into two categories, sparse-based retrieval (Bai et al, 2020;Shen et al, 2022;Formal et al, 2021;Gao et al, 2021), and dense-based retrieval Khattab and Zaharia, 2020;Zhan et al, 2021;Qiu et al, 2022;. Although there are differences in representation, they all learn the deep model through an end-to-end paradigm using contrastive learning methods.…”