Document reranking by term distribution and maximal marginal relevance for chinese information retrieval

Yang, Lingpeng; Ji, Donghong; Leong, Mun-Kew

doi:10.1016/j.ipm.2006.07.011

Cited by 7 publications

(5 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Maximal Marginal Relevance (MMR): MMR stands for Maximal Marginal Relevance. It's a technique used in information retrieval to select a subset of relevant documents from a larger set of search results [ 41 ]. MMR prioritizes the difference and diversity of the document group.…”

Section: Methodsmentioning

confidence: 99%

“…KeyBERT: It is a Python library that uses the BERT language model to extract keywords or keyphrases from a given text. It's a keyphrase extraction algorithm that is based on fine-tuning a BERT model on the specific task of keyphrase extraction [ 42 ]. In BERTopic, keyBERT is used to extract the most relevant and informative words or phrases from the documents in the corpus and to represent the documents in a high-dimensional vector space.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Exploring climate change discourse on social media and blogs using a topic modeling analysis

Gokcimen,

Das

2024

Heliyon

View full text Add to dashboard Cite

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Exploring climate change discourse on social media and blogs using a topic modeling analysis

Gokcimen,

Das

2024

Heliyon

View full text Add to dashboard Cite

“…This paper uses TextRank, TF-IDF, MMR, and LDA for key sentence extraction from four perspectives: graph-based ranking, statistical-based, maximum-edge-correlation-based, and topic-model-based. TextRank [28] and TF-IDF [29] are utilized frequently for keyword extraction, MMR [30] is utilized for document reordering, and LDA [31] is utilized for topic clustering. In principle, all of these methods compute the similarity between words or sentences, and key sentence extraction can be implemented based on the algorithm's underlying principles.…”

Section: Data Cleaningmentioning

confidence: 99%

Applicability Analysis and Ensemble Application of BERT with TF-IDF, TextRank, MMR, and LDA for Topic Classification Based on Flood-Related VGI

Yao

et al. 2023

IJGI

View full text Add to dashboard Cite

Volunteered geographic information (VGI) plays an increasingly crucial role in flash floods. However, topic classification and spatiotemporal analysis are complicated by the various expressions and lengths of social media textual data. This paper conducted applicability analysis on bidirectional encoder representation from transformers (BERT) and four traditional methods, TextRank, term frequency–inverse document frequency (TF-IDF), maximal marginal relevance (MMR), and linear discriminant analysis (LDA), and the results show that for user type, BERT performs best on the Government Affairs Microblog, whereas LDA-BERT performs best on the We Media Microblog. As for text length, TF-IDF-BERT works better for texts with a length of <70 and length >140 words, and LDA-BERT performs best with a text length of 70–140 words. For the spatiotemporal evolution pattern, the study suggests that in a Henan rainstorm, the textual topics follow the general pattern of “situation-tips-rescue”. Moreover, this paper detected the hotspot of “Metro Line 5” related to a Henan rainstorm and discovered that the topical focus of the Henan rainstorm spatially shifts from Zhengzhou, first to Xinxiang, and then to Hebi, showing a remarkable tendency from south to north, which was the same as the report issued by the authorities. We integrated multi-methods to improve the overall topic classification accuracy of Sina microblogs, facilitating the spatiotemporal analysis of flooding.

show abstract

“…Maximal Marginal Relevance (MMR) model is a method of redetermining document ranking values, which can be defined as a measure of the correlation between query and document and increase the diversity of documents retrieved by information retrieval systems . A series of algorithms have been proposed based on this ranking model. The MMR model, so to speak, has been widely applied as a benchmark for designing the construction of diverse ranking models.…”

Section: Related Workmentioning

confidence: 99%

“…22 In many studies, Machine learning methods have been optimized on training datasets, such as Perceptron Algorithm using Measures as Margins (PAMM), 23 Semi-Markov Conditional Random Fields (SMCRF), 24 and Search-based structured prediction (SEARN), 25 while the heuristic approaches construct which can be defined as a measure of the correlation between query and document and increase the diversity of documents retrieved by information retrieval systems. 26 A series of algorithms have been proposed [27][28][29] based on this ranking model. The MMR model, so to speak, has been widely applied as a benchmark for designing the construction of diverse ranking models.…”

Section: Related Workmentioning

confidence: 99%

Verifying the claimed sale‐ranking trustworthy: A maximum marginal relevance‐based ranking method

Wang

Fang

Shen

et al. 2019

Concurrency and Computation

View full text Add to dashboard Cite

Summary Various online contents on Internet platforms or search engines are related to the corporate reputation. Facing the huge amount of online contents, we need a mining method that can automatically extract and analyze a large number of network‐related information and obtain the real reliability of aspect for the content claimed by companies. In this paper, we propose to generate a ranking model to verify whether the sales‐rankings claimed by companies are trustworthy. The key idea is that the company that has higher confidence score should be supported by the online media. We use a unique data set of public opinion data related with a specific company, which we supplement with data from various online news platform and retrieval webpages using a distributed and generic Web crawler. Meanwhile, basic information and open financial data of companies are also collected for auxiliary analysis. We present a Maximal Marginal Relevance‐based ranking model to compute the confidence score of each company, taking into consideration the two technologies of word embedding and KL‐Divergence to filter the irrelevant documents. Extensive experiments show that the proposed method outperforms the state‐of‐the‐art MMR‐based method, and we showcase three representative cases about the corporate reputation built by us that gives positive, neutral, and negative support respectively to the sales‐ranking claim of companies.

show abstract

Document reranking by term distribution and maximal marginal relevance for chinese information retrieval

Cited by 7 publications

References 7 publications

Exploring climate change discourse on social media and blogs using a topic modeling analysis

Exploring climate change discourse on social media and blogs using a topic modeling analysis

Applicability Analysis and Ensemble Application of BERT with TF-IDF, TextRank, MMR, and LDA for Topic Classification Based on Flood-Related VGI

Verifying the claimed sale‐ranking trustworthy: A maximum marginal relevance‐based ranking method

Contact Info

Product

Resources

About