Towards information retrieval evaluation with reduced and only positive judgements

Mollá, Diego; Martínez, David; Amini, Iman

doi:10.1145/2537734.2537748

Cited by 2 publications

(2 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 3 shows that with a cutoff percentage of 80%, almost 24% of the documents considered relevant were actually judged relevant by human assessors and therefore we used this set (S) in the remainder of the experiment to expand the first set of qrels generated and judge more documents as relevant using the distance measure in equation (1). Relevant documents are at a close distance to each other, and in a sense they form a cluster [13]. Now that we have considered the documents retrieved by 80% of the systems as relevant, we tried to judge more documents in the pool of depth 100 as relevant based on the distance measure in (1).…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Towards automatic generation of relevance judgments for a test collection

Makary

Oakes

Yamout

2016

2016 Eleventh International Conference on Digital Information Management (ICDIM)

View full text Add to dashboard Cite

Abstract-this paper represents a new technique for building a relevance judgment list for information retrieval test collections without any human intervention. It is based on the number of occurrences of the documents in runs retrieved from several information retrieval systems and a distance based measure between the documents. The effectiveness of the technique is evaluated by computing the correlation between the ranking of the TREC systems using the original relevance judgment list (qrels) built by human assessors and the ranking obtained by using the newly generated qrels.

show abstract

Section: Resultsmentioning

confidence: 99%

“…Mollá [13] used a distance based measure to expand positive judgments only. The distance measure was based on the cosine similarity measure [15] between two document vectors.…”

Section: B Mollá's Techniquementioning

confidence: 99%