An information-theoretic perspective of tf–idf measures

Aizawa, Akiko

doi:10.1016/s0306-4573(02)00021-3

Cited by 1,086 publications

(493 citation statements)

References 14 publications

Supporting

Mentioning

486

Contrasting

Unclassified

Order By: Relevance

“…For documents, a Fisher kernel measures how much two members of a collection are similar taking into account a whole corpus as background information. We show that the Fisher kernel based on the DCM has a mathematical form related to the well-known TF-IDF representation for documents [1]. This demonstration is a new approach towards explaining why the TF-IDF heuristic is justified and why it is so successful experimentally.…”

Section: Introductionmentioning

confidence: 75%

Deriving TF-IDF as a Fisher Kernel

Elkan

2005

String Processing and Information Retrieval

View full text Add to dashboard Cite

Abstract. The Dirichlet compound multinomial (DCM) distribution has recently been shown to be a good model for documents because it captures the phenomenon of word burstiness, unlike standard models such as the multinomial distribution. This paper investigates the DCM Fisher kernel, a function for comparing documents derived from the DCM. We show that the DCM Fisher kernel has components that are similar to the term frequency (TF) and inverse document frequency (IDF) factors of the standard TF-IDF method for representing documents. Experiments show that the DCM Fisher kernel performs better than alternative kernels for nearest-neighbor document classification, but that the TF-IDF representation still performs best.

show abstract

Section: Introductionmentioning

confidence: 75%

Deriving TF-IDF as a Fisher Kernel

Elkan

2005

String Processing and Information Retrieval

View full text Add to dashboard Cite

show abstract

“…performansı artırmak için terimlere ağırlık atanmasıdır. Bu ağırlıklandırma işlemi ile bir terimin bir dokümandaki önemi belirtilmiş olur [5]. Bu nedenle, doküman temsilinin doğru ve etkili olmasında terim ağırlıklandırma önemli bir role sahiptir.…”

Section: İlgili çAlışmalarunclassified

The impact of term weighting method on Twitter sentiment analysis

Çoban¹,

Özyer²

2018

Pamukkale J Eng Sci

View full text Add to dashboard Cite

“…In order to select the links in a graph that are most relevant based on the given start and destination nodes, we utilize an adapted variant of the TF/IDF [1] measure: PF/IRF. The PF/IRF measure reflects the importance of a predicate with respect to a resource in a dataset and is defined as follows:…”

Section: Domain Delineationmentioning

confidence: 99%