Distance between sets as an objective measure of retrieval effectiveness

Heine, M.H.

doi:10.1016/0020-0271(73)90066-1

Cited by 51 publications

(14 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…[8]. Examples of other bivariate measures are 'Euclidean Distance' and 'Retrieval Power', introduced by the author in [1] and [9] respectively.~ The effect of query term specificity on search effectiveness has been discussed by Sparck Jones and others (e.g. [13]), and forms an integral part of discussions of related work by Miller [14], Robertson and Sparck Jones [15] Salton, Yang and Yu [16], Sparck Jones [17] and others on 'index term weighting'.…”

Section: Introductionmentioning

confidence: 99%

“…9 The assessment they offer of the relative influences of the search variables 'query size' and 'query term specificity' on retrieval effectiveness. 9 The comparison offered between the influences of (i) the search variables noted, and (ii) the contextual variable known as (information) 'generality' which identifies variability in the informing power of the database. o Their dependence on a method that operationalises relevance-decisions directly from realworld settings, rather than in one in which decisions by 'judges of relevance' are made in a laboratory-like setting.…”

Section: Introductionmentioning

confidence: 99%

“…In particular, these include discussion of the problems of evaluating retrieval effectiveness, as do specialist reviews cited in [1]. The bivariate (R-P) performance measure used in this study was introduced by the author in [9], and has been further discussed by Bollman and Cherniavsky [ 10] and Shaw [ 11 ] . Shaw favourably compared MZ with a related measure introduced earlier by Jardine and van Rijsbergen [12], discussed further by van Rijsbergen, e.g.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An investigation of the relative influences of database informativeness, query size and query term specificity on the effectiveness of Medline searching

Heine

1995

Journal of Information Science

Self Cite

View full text Add to dashboard Cite

An investigation of search effectiveness in Medline used regression analysis to compare the relative influences on search performance (characterised by Recall, Precision and the MZ-metric) of the three variables: database informativeness, and size and mean specificity of the query. (A 'query' is defined as an unstructured set of search terms.) A reduction in random and/or systematic error was sought by means of three experimental devices: (i) error arising from the subjectivity of third-party relevance judges was minimised by using behaviourally-generated target sets of records; (ii) errors associated with the definitions of queries were avoided (or at least standardised) by using algorithmically derived queries, and (iii) error associated with uncontrolled variation in the logical structure of the search expression was minimised by using search expressions that had been optimised separately against each of the three performance criteria referred to. It was found that the dominant variable influencing search performance was the number of query terms. Database informativeness had a statistically significant, but small, influence on search performance in almost all the data-sets, and this influence could be in either direction, depending on the data-set. The influence of term specificity was in general not statistically significant. Overall, the three predicting variables were able to explain up to 50% of the variation in search performance.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An investigation of the relative influences of database informativeness, query size and query term specificity on the effectiveness of Medline searching

Heine

1995

Journal of Information Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Via this e measure, the ENIRR is (positively) linearly related to the rank correlation coefficient [Kendall 1955] between two linear orders by rewriting it as 1 -2 ·e(n, q) = E (N ) (n, q) · p (q, r)/[α · χ · n + (1 − α) · χ · R(q)] − 1. Heine [1973] proposed to use the Marczewski-Steinhaus metric [Marczewski and Steinhaus 1958] to evaluate retrieval systems. This metric is a distance measure that is related to NDPM.…”

Section: Normalized Distance-based Performance Measurementioning

confidence: 99%

Building a framework for the probability ranking principle by a family of expected weighted rank

Dang

Luk

et al. 2009

ACM Trans. Inf. Syst.

View full text Add to dashboard Cite

A new principles framework is presented for retrieval evaluation of ranked outputs. It applies decision theory to model relevance decision preferences and shows that the Probability Ranking Principle (PRP) specifies optimal ranking. It has two new components, namely a probabilistic evaluation model and a general measure of retrieval effectiveness. Its probabilities may be interpreted as subjective or objective ones. Its performance measure is the expected weighted rank which is the weighted average rank of a retrieval list. Starting from this measure, the expected forward rank and some existing retrieval effectiveness measures (e.g., top n precision and discounted cumulative gain) are instantiated using suitable weighting schemes after making certain assumptions. The significance of these instantiations is that the ranking prescribed by PRP is shown to be optimal simultaneously for all these existing performance measures. In addition, the optimal expected weighted rank may be used to normalize the expected weighted rank of retrieval systems for (summary) performance comparison (across different topics) between systems. The framework also extends PRP and our evaluation model to handle graded relevance, thereby generalizing the discussed, existing measures (e.g., top n precision) and probabilistic retrieval models for graded relevance.

show abstract

“…This measure is also widely used in Information Retrieval and its calculation formula differs from author to author [5,10,13,15]. For the need of this work we have chosen the definition proposed in [15]:…”

Section: Definitionmentioning

confidence: 99%

A fuzzy-based method for improving recall values in recommender systems

Kiewra

Nguyên

2009

Journal of Intelligent &Amp; Fuzzy Systems

View full text Add to dashboard Cite

In this paper a fuzzy-based recommendation method is presented. Its main goal is to improve the recommendation recall maintaining high recommendation precision. The formal model has been built to describe the method and to analyze how the measures used in traditional Information Retrieval may be adapted to evaluate the effectiveness of recommendation process. The original contributions consist among others of proving several properties which show that the method is able to adapt to changing user's needs and achieving the maximum effectiveness if the component methods work properly.

show abstract

Distance between sets as an objective measure of retrieval effectiveness

Cited by 51 publications

References 14 publications

An investigation of the relative influences of database informativeness, query size and query term specificity on the effectiveness of Medline searching

An investigation of the relative influences of database informativeness, query size and query term specificity on the effectiveness of Medline searching

Building a framework for the probability ranking principle by a family of expected weighted rank

A fuzzy-based method for improving recall values in recommender systems

Contact Info

Product

Resources

About