Unsupervised Document Expansion for Information Retrieval with Stochastic Text Generation

Abstract

One of the challenges in information retrieval (IR) is the vocabulary mismatch problem, which happens when the terms between queries and documents are lexically different but semantically similar. While recent work has proposed to expand the queries or documents by enriching their representations with additional relevant terms to address this challenge, they usually require a large volume of query-document pairs to train an expansion model. In this paper, we propose an Unsupervised Document Expansion with Generation (UDEG) framework with a pretrained language model, which generates diverse supplementary sentences for the original document without using labels on querydocument pairs for training. For generating sentences, we further stochastically perturb their embeddings to generate more diverse sentences for document expansion. We validate our framework on two standard IR benchmark datasets. The results show that our framework significantly outperforms relevant expansion baselines for IR.

LoGE: an unsupervised local-global document extension generation in information retrieval for long documents

“…We compare LoGE with: The Base contains only the retrieval by BM25 scoring on documents preprocessed with the basic filters; The Pegasus4IR model, for which we have adapted Pegasus (Zhang et al , 2020), is for IR. We generate a rewritten text by applying the most widely used pretrained Pegasus model in document summarization; and UDEG (Jeong et al , 2021) is the state-of-the-art model for abstractive generation of document extensions for ad hoc search. It is the main competitive model with a similar approach to unsupervised document extension. …”

“…UDEG (Jeong et al , 2021) is the state-of-the-art model for abstractive generation of document extensions for ad hoc search. It is the main competitive model with a similar approach to unsupervised document extension.…”

“…Some works focus on resolving those problems by using deep learning for IR in the absence of query/document pairs. We can cite the UDEG model (Jeong et al , 2021). This one uses an abstract automatic summarization system to summarize documents (Lu and Conrad, 2012; Koh et al , 2022).…”

Augmenting Document Representations for Dense Retrieval with Interpolation and Perturbation

“…Besides interpolation, Wei and Zou (2019) and Ma (2019) proposed perturbation over words, and Lee et al (2021b) proposed perturbation over word embeddings. Jeong et al (2021) and Gao et al (2021) perturbed text embeddings to generate diverse sentences and to augment positive sentence pairs in unsupervised learning. In contrast, we address dense retrieval, perturbing document representations with dropout (Srivastava et al, 2014) in a supervised setting with labeled documents.…”