Term weighting based on document revision history

Nunes, Sérgio; Ribeiro, Cristina; David, Gabriel

doi:10.1002/asi.21597

Cited by 9 publications

(8 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The aim is to extract the significant terms from a set of document versions which bring changed and novel information into an article. Research has been done on term weighting using the time impact on changes occurring either in the collection (Efron, 2010) or in an individual document (Nunes, Ribeiro, & David, 2011). Instead of directly taking into account the periods of time in which a term occurs throughout an article's revision history, the score of a term is calculated here by considering the joint probabilities of both the insertion and deletion events occurring in a set of document versions.…”

Section: Approach-ii: Temporal Sentence Score (Tss)mentioning

confidence: 99%

Summarization of changes in dynamic text collections using Latent Dirichlet Allocation model

Kar

Nunes²,

Ribeiro³

2015

Information Processing & Management

Self Cite

View full text Add to dashboard Cite

a b s t r a c tIn the area of Information Retrieval, the task of automatic text summarization usually assumes a static underlying collection of documents, disregarding the temporal dimension of each document. However, in real world settings, collections and individual documents rarely stay unchanged over time. The World Wide Web is a prime example of a collection where information changes both frequently and significantly over time, with documents being added, modified or just deleted at different times. In this context, previous work addressing the summarization of web documents has simply discarded the dynamic nature of the web, considering only the latest published version of each individual document. This paper proposes and addresses a new challenge -the automatic summarization of changes in dynamic text collections. In standard text summarization, retrieval techniques present a summary to the user by capturing the major points expressed in the most recent version of an entire document in a condensed form. In this new task, the goal is to obtain a summary that describes the most significant changes made to a document during a given period. In other words, the idea is to have a summary of the revisions made to a document over a specific period of time. This paper proposes different approaches to generate summaries using extractive summarization techniques. First, individual terms are scored and then this information is used to rank and select sentences to produce the final summary. A system based on Latent Dirichlet Allocation model (LDA) is used to find the hidden topic structures of changes. The purpose of using the LDA model is to identify separate topics where the changed terms from each topic are likely to carry at least one significant change. The different approaches are then compared with the previous work in this area. A collection of articles from Wikipedia, including their revision history, is used to evaluate the proposed system. For each article, a temporal interval and a reference summary from the article's content are selected manually. The articles and intervals in which a significant event occurred are carefully selected. The summaries produced by each of the approaches are evaluated comparatively to the manual summaries using ROUGE metrics. It is observed that the approach using the LDA model outperforms all the other approaches. Statistical tests reveal that the differences in ROUGE scores for the LDA-based approach is statistically significant at 99% over baseline.

show abstract

Section: Approach-ii: Temporal Sentence Score (Tss)mentioning

confidence: 99%

Summarization of changes in dynamic text collections using Latent Dirichlet Allocation model

Kar

Nunes²,

Ribeiro³

2015

Information Processing & Management

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, the proposed model was shown to be outperformed by the baselines for ambiguous queries which spans multiple topics such as the query "earthquake prediction" that include geology, natural disasters, etc. Following the same direction, Nunes et al [48] used the same document's revision history as a source of temporal evidence and propose several different term-weighting measures. The basic idea of the approach is to give higher weight values to terms that have existed for a longer time in a document, since its first version should be valued higher than a term that was introduced only in the latest revision made.…”

Section: Time At the Ranking Levelmentioning

confidence: 99%

When time meets information retrieval: Past proposals, current plans and future trends

Moulahi

Tamine

Yahia

2016

Journal of Information Science

View full text Add to dashboard Cite

With the advent of Web search and the large amount of data published on the Web sphere, a tremendous amount of documents become strongly time-dependent. In this respect, the time dimension has been extensively exploited as a highly important relevance criterion to improve the retrieval effectiveness of document ranking models. Thus, a compelling research interest is going on the temporal information retrieval realm, which gives rise to several temporal search applications. In this article, we intend to provide a scrutinizing overview of time-aware information retrieval models. We specifically put the focus on the use of timeliness and its impact on the global value of relevance as well as on the retrieval effectiveness. First, we attempt to motivate the importance of temporal signals, whenever combined with other relevance features, in accounting for document relevance. Then, we review the relevant studies standing at the crossroads of both information retrieval and time according to three common information retrieval aspects: the query level, the document content level and the document ranking model level. We organize the related temporalbased approaches around specific information retrieval tasks and regarding the task at hand, we emphasize the importance of results presentation and particularly timelines to the end user. We also report a set of relevant research trends and avenues that can be explored in the future.

show abstract

“…Reference [8] (12) In this measure the impact of the other terms of document d has been considered, too.…”

Section: Term Weighting Based On Document Revision Histroymentioning

confidence: 99%

“…For example, the Wikipedia pages have different versions that these versions are created by different people to improve the content of the pages. Previous researches [1][2][3][4][5][6][7][8][9][10][11] have shown that investigating on these changes can improve the efficiency of information retrieval systems.…”

Section: Introductionmentioning

confidence: 99%

Improving the Efficiency of Term Weighting in Set of Dynamic Documents

Jabalameli

Arman²,

Nematbakhsh³

2015

IJMECS

View full text Add to dashboard Cite

In real information systems, there are few static documents. On the other hand, there are too many documents that their content change during the time that could be considered as signals to improve the quality of information retrieval. Unfortunately, considering all these changes could be time-consuming. In this paper, a method has been proposed that the time of analyzing these changes could be reduced significantly. The main idea of this method is choosing a special part of changes that do not make effective changes in the quality of information retrieval; but it could be possible to reduce the analyzing time. To evaluate the proposed method, three different datasets selected from Wikipedia. Different factors have been assessed in term weighting and the effect of the proposed method investigated on these factors. The results of empirical experiments showed that the proposed method could keep the quality of retrieved information in an acceptable rate and reduce the documents' analysis time as a result.

show abstract

Term weighting based on document revision history

Cited by 9 publications

References 10 publications

Summarization of changes in dynamic text collections using Latent Dirichlet Allocation model

Summarization of changes in dynamic text collections using Latent Dirichlet Allocation model

When time meets information retrieval: Past proposals, current plans and future trends

Improving the Efficiency of Term Weighting in Set of Dynamic Documents

Contact Info

Product

Resources

About