A recursive TF-ISF Based Sentence Retrieval Method with Local Context

Abstract

Abstract-Sentence retrieval consists of retrieving relevant sentences from a document base in response to a query. Question answering, novelty detection, summarization, opinion mining and information provenance make use of sentence retrieval. Most of the sentence retrieval methods are trivial adaptations of document retrieval methods. However some newer sentence retrieval methods based on the language modeling framework successfully use some kind of context of sentences. Unlike that there is no successful improvement of the TF-ISF method that takes into account the context of sentences. In this paper we propose a recursive TF-ISF based method that takes into account the local context of a sentence. The context is considered the previous and next sentence of current sentence. We compared the new method to the TF-ISF baseline and to an earlier unsuccessful method that also incorporates a similar context into TF-ISF. We got statistically significant improvements of the results in comparison to both of the methods. Additional benefit of our method is the clear explicit model of the context that will allow us to automatically generate a document representation with context suitable for sentence retrieval which is important for our future work.

Learning to Explain Entity Relationships in Knowledge Graphs

“…where tf t,q and tf t,s are the number of occurrences of term t in the query q and the sentence s respectively, sf t is the number of sentences in which t appears, and n is the number of sentences in the collection. R-TFISF is an improved extension of the TFISF method (Doko et al, 2013), which incorporates context from neighboring sentences in the ranking function:…”

“…Previous research shows that using surrounding sentences is beneficial for sentence retrieval (Doko et al, 2013). We therefore consider the number of common links in the previous and next sentence (features 29-30).…”

“…Document retrieval models such as tf-idf, BM25, and language modeling (Baeza-Yates et al, 1999) have been extended to tackle sentence retrieval. Three of the most successful sentence retrieval methods are TFISF (Allan et al, 2003), which is a variant of the vector space model with tf-idf weighting, language modeling with local context (Murdock, 2006;Fernández et al, 2011), and a recursive version of TFISF that accounts for local context (Doko et al, 2013). TFISF is very competitive compared to document retrieval models tuned specifically for sentence retrieval (e.g., BM25 and language modeling (Losada, 2008)) and we therefore include it as a baseline.…”

“…R-TFISF is an improved extension of the TFISF method (Doko et al, 2013), which incorporates context from neighboring sentences in the ranking function:…”

Developing a Collaborative MOOC Learning Environment utilizing Video Sharing with Discussion Summarization as Added-Value

“…The text in these tools is usually not or semi-structured, and falls under the definition of microtext [1]. Therefore, the summarization process may require different techniques and approaches like the ones in [2], [3], [4], and [5]. We have followed sentence extraction summarization technique with different sentence ranking approach than in [4].…”