A Tri-Partite Neural Document Language Model for Semantic Information Retrieval

Nguyen, Gia-Hung; Tamine, Lynda; Soulier, Laure; Souf, Nathalie

doi:10.1007/978-3-319-93417-4_29

Cited by 7 publications

(14 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The learned representations are injected in a text-to-text matching process according to a query expansion technique. Nguyen et al [48] propose a tri-partite neural language model that leverages explicit knowledge to jointly constrain word, concept, and document representations. The authors employ the model in two retrieval strategies: document re-ranking and query expansion.…”

Section: Knowledge-enhanced Representation Modelsmentioning

confidence: 99%

“…The authors employ the model in two retrieval strategies: document re-ranking and query expansion. Tamine et al [66] extend [47,48] to investigate the combined use of corpus-based information and external knowledge resources in different NLP and IR tasks. The authors compare the impact of the different learning approaches on the quality of the learned representations.…”

Section: Knowledge-enhanced Representation Modelsmentioning

confidence: 99%

“…To this end, recent works integrate external knowledge into the learning process of neural language models to reduce the effect of the semantic gap between queries and documents [42,47,48]. However, even though knowledge-enhanced language models have been proven effective in many Natural Language Processing (NLP) tasks, their effectiveness is limited in IR [66].…”

Section: Introductionmentioning

confidence: 99%

“…We identify two reasons causing this performance gap. First, knowledge-enhanced language models have been used in IR mostly for re-ranking [42,48]. In re-ranking, knowledge-enhanced language Learning Unsupervised Knowledge-Enhanced Representations to Reduce 38:3 models are limited to candidate documents retrieved by traditional bag-of-words models, which are not suited to address the semantic gap.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Learning Unsupervised Knowledge-Enhanced Representations to Reduce the Semantic Gap in Information Retrieval

Agosti

Marchesin

Silvello

2020

ACM Trans. Inf. Syst.

View full text Add to dashboard Cite

The semantic mismatch between query and document terms-i.e., the semantic gap-is a long-standing problem in Information Retrieval (IR). Two main linguistic features related to the semantic gap that can be exploited to improve retrieval are synonymy and polysemy. Recent works integrate knowledge from curated external resources into the learning process of neural language models to reduce the effect of the semantic gap. However, these knowledge-enhanced language models have been used in IR mostly for re-ranking and not directly for document retrieval. We propose the Semantic-Aware Neural Framework for IR (SAFIR), an unsupervised knowledge-enhanced neural framework explicitly tailored for IR. SAFIR jointly learns word, concept, and document representations from scratch. The learned representations encode both polysemy and synonymy to address the semantic gap. SAFIR can be employed in any domain where external knowledge resources are available. We investigate its application in the medical domain where the semantic gap is prominent and there are many specialized and manually curated knowledge resources. The evaluation on shared test collections for medical literature retrieval shows the effectiveness of SAFIR in terms of retrieving and ranking relevant documents most affected by the semantic gap.

show abstract

Section: Knowledge-enhanced Representation Modelsmentioning

confidence: 99%

Section: Knowledge-enhanced Representation Modelsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learning Unsupervised Knowledge-Enhanced Representations to Reduce the Semantic Gap in Information Retrieval

Agosti

Marchesin

Silvello

2020

ACM Trans. Inf. Syst.

View full text Add to dashboard Cite

show abstract

“…However, these models suffer from two main limitations: they fail to discriminate polysemous words, as the different meanings of a word are conflated into a single representation; and they fail to learn close representations for synonyms occurring in different contexts, as they lack the relational knowledge required to identify synonymy relationships between words. To overcome these limitations, recent works that integrate external knowledge into the learning process of neural models have been proposed in the Natural Language Processing (NLP) community, but only a few have been applied in IR to reduce the effect of the semantic gap between queries and documents [147,169,168,170].…”

mentioning

confidence: 99%

Developing unsupervised knowledge-enhanced models to reduce the semantic gap in information retrieval

Marchesin

2021

SIGIR Forum

View full text Add to dashboard Cite

In this thesis we tackle the semantic gap, a long-standing problem in Information Retrieval (IR). The semantic gap can be described as the mismatch between users' queries and the way retrieval models answer to such queries. Two main lines of work have emerged over the years to bridge the semantic gap: (i) the use of external knowledge resources to enhance the bag-of-words representations used by lexical models, and (ii) the use of semantic models to perform matching between the latent representations of queries and documents. To deal with this issue, we first perform an in-depth evaluation of lexical and semantic models through different analyses [Marchesin et al., 2019]. The objective of this evaluation is to understand what features lexical and semantic models share, if their signals are complementary, and how they can be combined to effectively address the semantic gap. In particular, the evaluation focuses on (semantic) neural models and their critical aspects. Each analysis brings a different perspective in the understanding of semantic models and their relation with lexical models. The outcomes of this evaluation highlight the differences between lexical and semantic signals, and the need to combine them at the early stages of the IR pipeline to effectively address the semantic gap. Then, we build on the insights of this evaluation to develop lexical and semantic models addressing the semantic gap. Specifically, we develop unsupervised models that integrate knowledge from external resources, and we evaluate them for the medical domain - a domain with a high social value, where the semantic gap is prominent, and the large presence of authoritative knowledge resources allows us to explore effective ways to address it. For lexical models, we investigate how - and to what extent - concepts and relations stored within knowledge resources can be integrated in query representations to improve the effectiveness of lexical models. Thus, we propose and evaluate several knowledge-based query expansion and reduction techniques [Agosti et al., 2018, 2019; Di Nunzio et al., 2019]. These query reformulations are used to increase the probability of retrieving relevant documents by adding to or removing from the original query highly specific terms. The experimental analyses on different test collections for Precision Medicine - a particular use case of Clinical Decision Support (CDS) - show the effectiveness of the proposed query reformulations. In particular, a specific subset of query reformulations allow lexical models to achieve top performing results in all the considered collections. Regarding semantic models, we first analyze the limitations of the knowledge-enhanced neural models presented in the literature. Then, to overcome these limitations, we propose SAFIR [Agosti et al., 2020], an unsupervised knowledge-enhanced neural framework for IR. SAFIR integrates external knowledge in the learning process of neural IR models and it does not require labeled data for training. Thus, the representations learned within this framework are optimized for IR and encode linguistic features that are relevant to address the semantic gap. The evaluation on different test collections for CDS demonstrate the effectiveness of SAFIR when used to perform retrieval over the entire document collection or to retrieve documents for Pseudo Relevance Feedback (PRF) methods - that is, when it is used at the early stages of the IR pipeline. In particular, the quantitative and qualitative analyses highlight the ability of SAFIR to retrieve relevant documents affected by the semantic gap, as well as the effectiveness of combining lexical and semantic models at the early stages of the IR pipeline - where the complementary signals they provide can be used to obtain better answers to semantically hard queries.

show abstract

Deep learning based Bengali question answering system using semantic textual similarity

Das

Saha

2021

Multimed Tools Appl

View full text Add to dashboard Cite

A Tri-Partite Neural Document Language Model for Semantic Information Retrieval

Cited by 7 publications

References 28 publications

Learning Unsupervised Knowledge-Enhanced Representations to Reduce the Semantic Gap in Information Retrieval

Learning Unsupervised Knowledge-Enhanced Representations to Reduce the Semantic Gap in Information Retrieval

Developing unsupervised knowledge-enhanced models to reduce the semantic gap in information retrieval

Deep learning based Bengali question answering system using semantic textual similarity

Contact Info

Product

Resources

About