On Relevance, Probabilistic Indexing and Information Retrieval

Abstract

This paper reports on a novel technique for literature indexing and searching in a mechanized library system. The notion of relevance is taken as the key concept in the theory of information retrieval and a comparative concept of relevance is explicated in terms of the theory of probability. The resulting technique called “Probabilistic Indexing,” allows a computing machine, given a request for information, to make a statistical inference and derive a number (called the “relevance number”) for each document, which is a measure of the probability that the document will satisfy the given request. The result of a search is an ordered list of those documents which satisfy the request ranked according to their probable relevance. The paper goes on to show that whereas in a conventional library system the cross-referencing (“see” and “see also”) is based solely on the “semantical closeness” between index terms, statistical measures of closeness between index terms can be defined and computed. Thus, given an arbitrary request consisting of one (or many) index term(s), a machine can elaborate on it to increase the probability of selecting relevant documents that would not otherwise have been selected. Finally, the paper suggests an interpretation of the whole library problem as one where the request is considered as a clue on the basis of which the library system makes a concatenated statistical inference in order to provide as an output an ordered list of those documents which most probably satisfy the information needs of the user.

Information Retrieval Models

“…As early as 1960, Bill Maron and Larry Kuhns (Maron and Kuhns 1960) defined their probabilistic indexing model. Unlike Luhn, they did not target automatic indexing by information retrieval systems.…”

Systematic acceleration of radical discovery and innovation in science and technology

“…Studies related to the use of Term Co-occurrence in information retrieval can be traced back to at least the 1960s (Maron and Kuhns, 1960;Stiles;1961;Lesk, 1969). These early experiments demonstrated the potential of Term Co-occurrence data for the identification of search term variants.…”

Bubble World--A Novel Visual Information Retrieval Technique

“…Maron and Kuhns [12] argued against Luhn's ideas on using similarity, and instead suggest that the documents in a collection should be ranked according to their probability of relevance. Robertson [13] called this criterion the 'probability ranking principle' and formulated the following principle:…”