A Comparative Analysis of Sentence Embedding Techniques for Document Ranking

Abstract

Due to the exponential increase in the information on the web, extracting relevant documents for users in a reasonable time becomes a cumbersome task. Also, when user feedback is scarce or unavailable, content-based approaches to extract and rank relevant documents are critical as they suffer from the problem of determining semantic similarity between texts of user queries and documents. Various sentence embedding models exist today that acquire deep semantic representations through training on a large corpus, with the goal of providing transfer learning to a broad range of natural language processing tasks such as document similarity, text summarization, text classification, sentiment analysis, etc. So, in this paper, a comparative analysis of six pre-trained sentence embedding techniques has been done to identify the best model suited for document ranking in IR systems. These are SentenceBERT, Universal Sentence Encoder, InferSent, ELMo, XLNet, and Doc2Vec. Four standard datasets CACM, CISI, ADI, and Medline are used to perform all the experiments. It is found that Universal Sentence Encoder and SentenceBERT outperform other techniques on all four datasets in terms of MAP, recall, F-measure, and NDCG. This comparative analysis offers a synthesis of existing work as a single point of entry for practitioners who seek to use pre-trained sentence embedding models for document ranking and for scholars who wish to undertake work in a similar domain. The work can be expanded in many directions in the future as various researchers can combine these strategies to build a hybrid document ranking system or query reformulation system in IR.

Improving Ranking Using Hybrid Custom Embedding Models on Persian Web

“…Additionally, in 2022, some pre-trained embedding methods were evaluated to identify the best model for document ranking. The evaluation results indicate better performance of the joint sentence encoder and SentenceBERT [34].…”

Embedding a Microblog Context in Ephemeral Queries for Document Retrieval

“…Retrieval of information from search repositories is highly dependent on keywords supplied by the user in its query. The pages are retrieved by matching the query keywords with the term weight vector of the documents archived in the search engine's database [20]. The scheme performs outstandingly when the user transforms their information needs into a well-defined set of keywords.…”

“…Recently, Chen et al proposed a hierarchical neural session-based method to capture user behavior and search context [9]. Google also introduced an open-sourced NLP technique named 'BERT' to dynamically understand the context of a user query [20]. Many blogs are also available; however, the exact algorithm is not still disclosed [22].…”