Accessibility in Information Retrieval

Azzopardi, Leif; Vinay, V.

doi:10.1007/978-3-540-78646-7_46

Cited by 24 publications

(24 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Retrievability was introduced to measure how likely a document is to be retrieved given an IR system [5][6][7]. Computing the retrievability scores requires the availability of a large query set, but without the need for relevance judgments.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Quantifying retrieval bias in Web archive search

et al. 2017

View full text Add to dashboard Cite

A Web archive usually contains multiple versions of documents crawled from the Web at different points in time. One possible way for users to access a Web archive is through full-text search systems. However, previous studies have shown that these systems can induce a bias, known as the retrievability bias, on the accessibility of documents in community-collected collections (such as TREC collections). This bias can be measured by analyzing the distribution of the retrievability scores for each document in a collection, quantifying the likelihood of a document's retrieval. We investigate the suitability of retrievability scores in retrieval systems that consider every version of a document in a Web archive as an independent document. We show that the retrievability of documents can vary for different versions of the same document and that retrieval systems induce biases to different extents. We quantify this bias for a retrieval system which is adapted to handle multiple versions of the same document. The retrieval system indexes each version of a document independently, and we refine the search results using two techniques to aggregate similar versions. The first approach is to collapse similar versions of a document based on content similarity. The second approach is to collapse all versions of the same document based on their URLs. In both cases, we found that the degree of bias Universiteit Utrecht, Utrecht, The Netherlands is related to the aggregation level of versions of the same document. Finally, we study the effect of bias across time using the retrievability measure. Specifically, we investigate whether the number of documents crawled in a particular year correlates with the number of documents in the search results from that year. Assuming queries are not inherently temporal in nature, the analysis is based on the timestamps of documents in the search results returned using the retrieval model for all queries. The results show a relation between the number of documents per year and the number of documents retrieved by the retrieval system from that year. We further investigated the relation between the queries' timestamps and the documents' timestamps. First, we split the queries into different time frames using a 1-year granularity. Then, we issued the queries against the retrieval system. The results show that temporal queries indeed retrieve more documents from the assumed time frame. Thus, the documents from the same time frame were preferred by the retrieval system over documents from other time frames.

show abstract

Section: Related Workmentioning

confidence: 99%

“…ing stage 5 and switch off stopword removal and stemming at indexing time (as these have already been applied in the preprocessing stage). The index granularity is the document's version d t i v .…”

Section: Preprocessing and Indexingmentioning

confidence: 99%

Quantifying retrieval bias in Web archive search

et al. 2017

View full text Add to dashboard Cite

show abstract

“…• Findability [30] • Navigability [41,21,23] • Accessibility [28,11] • Searchability [35] • Crawlability [29] • Discoverability [22] • Usability [32] • Retrievability [13] 2. What is retrievability?…”

Section: The -Abilities Of Information Retrievalmentioning

confidence: 99%

The retrievability of documents

Azzopardi

2014

Proceedings of the 37th International ACM SIGIR Conference on Research &Amp; Development in Information Retrieval

Self Cite

View full text Add to dashboard Cite

Retrievability is an important and interesting indicator that can be used in a number of ways to analyse Information Retrieval systems and document collections. Rather than focusing totally on relevance, retrievability examines what is retrieved, how often it is retrieved, and whether a user is likely to retrieve it or not. This is important because a document needs to be retrieved, before it can be judged for relevance. In this tutorial, we shall explain the concept of retrievability along with a number of retrievability measures, how it can be estimated and how it can be used for analysis. Since retrieval precedes relevance, we shall also provide an overview of how retrievability relates to effectivenessdescribing some of the insights that researchers have discovered thus far. We shall also show how retrievability relates to efficiency, and how the theory of retrievability can be used to improve both effectiveness and efficiency. Then we shall provide an overview of the different applications of retrievability such as Search Engine Bias, Corpus Profiling, etc., before wrapping up with challenges and opportunities. The final session of the day will look at example problems and ways to analyse and apply retrievability to other problems and domains. Participants are invited to bring their own problems to create a more focused practical session. This tutorial is ideal for: (i) researchers curious about retrievability and wanting to see how it can impact their research, (ii) researchers who would like to expand their set of analysis techniques, and/or (iii) researchers who would like to use retrievability to perform their own analysis.

show abstract

“…Retrievability is essentially a measure of the capabilities of a particular retrieval system to return documents in the collection, and retrieval bias is a measurement on the system about how the retrieval process makes certain documents more retrievable than others [4]. In prior work, it has been shown that the fairness of a system tends to correlate with retrieval performance [20], and that tuning the system via minimizing the retrieval bias may lead to improved performance [19].…”

Section: Introductionmentioning

confidence: 99%

An Empirical Analysis of Pruning Techniques

Chen

Azzopardi

Scholer

2017

Proceedings of the 2017 ACM on Conference on Information and Knowledge Management

Self Cite

View full text Add to dashboard Cite

Prior work on using retrievability measures in the evaluation of information retrieval (IR) systems has laid out the foundations for investigating the relation between retrieval performance and retrieval bias. While various factors in uencing retrievability have been examined, showing how the retrieval model may in uence bias, no prior work has examined the impact of the index (and how it is optimized) on retrieval bias. Intuitively, how the documents are represented, and what terms they contain, will in uence whether they are retrievable or not. In this paper, we investigate how the retrieval bias of a system changes as the inverted index is optimized for e ciency through static index pruning. In our analysis, we consider four pruning methods and examine how they a ect performance and bias on the TREC GOV2 Collection. Our results show that the relationship between these factors is varied and complexand very much dependent on the pruning algorithm. We nd that more pruning results in relatively little change or a slight decrease in bias up to a point, and then a dramatic increase. The increase in bias corresponds to a sharp decrease in early precision such as NDCG@10 and is also indicative of a large decrease in MAP. The ndings suggest that the impact of pruning algorithms can be quite varied -but retrieval bias could be used to guide the pruning process. Further work is required to determine precisely which documents are most a ected and how this impacts upon performance.

show abstract

Accessibility in Information Retrieval

Cited by 24 publications

References 9 publications

Quantifying retrieval bias in Web archive search

Quantifying retrieval bias in Web archive search

The retrievability of documents

An Empirical Analysis of Pruning Techniques

Contact Info

Product

Resources

About