A New Term Significance Weighting Approach

The influence of indexing practices and weighting algorithms on document spaces

“…These values were not available through ERIC but were modeled on other observed data. Through analysis of index term weight characteristics used in a previous study, Zhang and Nguyen (2005) found that different weighting algorithms produced different probability distributions of average weights for indexable terms of varying frequencies.…”

“…In this case, generated descriptor weights for each descriptor frequency across the document set were based on a lognormal distribution of a given mean (determined by the descriptor frequency) and standard deviation (fixed at 0.5 across all descriptor frequencies), resulting in a range of weights for each descriptor frequency across the document set. The use of a lognormal distribution, instead of a normal distribution of weights, more closely models the skewed distribution of term weights for each frequency as observed in the dataset used by Zhang and Nguyen (2005).…”

Optimized weights of document keywords for auto-reply accuracy

“…Term-frequency weights have been used for many years in automatic indexing environments. A term frequency (TF) was used as part of the term-weighting system measuring the frequency of occurrence of the terms in the document or query ( [4,22,23,35]). Term frequency alone cannot ensure acceptable retrieval performance, when the high frequency terms are not concentrated in a few particular documents.…”

A Novel Vector-based Keyword Extracting Tool

“…Two terms with the same width characteristics of term distribution will have different significance measures if their frequencies within the documents containing them are different. This paper describes a novel keyword extracting tool that is distinguished itself from existing ones by its efficient and effective keyword significance measure [2]. The measure integrates term frequency retrieval characteristics, document collection characteristics, and both the term width and depth distribution characteristics.…”

“…We assume that d k is not equal to zero, therefore, L k is not equal to zero either; when d k or L k is equal to zero, the corresponding W ik is defined as zero. Detailed analyses on this significance measure can be found in [2].…”