Theory of Retrieval

Abstract

Retrievability is an important and interesting indicator that can be used in a number of ways to analyse Information Retrieval systems and document collections. Rather than focusing totally on relevance, retrievability examines what is retrieved, how often it is retrieved, and whether a user is likely to retrieve it or not. This is important because a document needs to be retrieved, before it can be judged for relevance. In this tutorial, we shall explain the concept of retrievability along with a number of retrievability measures, how it can be estimated and how it can be used for analysis. Since retrieval precedes relevance, we shall also provide an overview of how retrievability relates to e↵ectiveness -describing some of the insights that researchers have discovered thus far. We shall also show how retrievability relates to e ciency, and how the theory of retrievability can be used to improve both e↵ectiveness and e ciency. Then we shall provide an overview of the di↵erent applications of retrievability such as Search Engine Bias, Corpus Profiling, etc., before wrapping up with challenges and opportunities. The final session will look at example problems and ways to analyse and apply retrievability to other problems and domains. Participants are invited to bring their own problems to be discussed after the tutorial. This half-day tutorial is ideal for: (i) researchers curious about retrievability and wanting to see how it can impact their research, (ii) researchers who would like to expand their set of analysis techniques, and/or (iii) researchers who would like to use retrievability to perform their own analysis.

Neural Passage Quality Estimation for Static Pruning

“…• BM25 » MonoELECTRA (Cross-Encoder). Finally, we re-rank the top 100 BM25 results using MonoELECTRA [46], 5 a strong cross-encoder model trained with hard negatives.…”

“…In contrast with prior work on static pruning, which focus on removing individual tokens [11] or token representations [1] from a passage, we investigate ways to remove passages in their entirety. Alternatively, this work can be viewed as an extension of retrievability [5], which studies the fairness of retrieval systems from a producer perspective by measuring which documents are less prone to retrieval. While we recognise the potential for a passage pruning approach to yield unfair results from a producer perspective, we argue that there are substantial benefits for the search engine operator and searcher perspectives, namely in reduced computational overheads.…”

Why is Information Retrieval a Scientific Discipline?

“…In Physics, Einstein's papers on special/general relativity are examples of theoretical studies. IR scientific study has theoretical studies too, e.g., Robertson (1977), Dang et al (2009), Lafferty and Zhai (2001), Azzopardi and Roelleke (2007) and Luk (2008).…”