Methodologies for distributed information retrieval

Abstract

Text collections have traditionally been located at a single site and managed as a monolithic whole. However, it is now common for a collection to be spread over several hosts and for these hosts to be geographically separated. In this paper we examine several alternative approaches to distributed text retrieval. We report on our experience with a full implementation of these methods, and give retrieval efficiency and retrieval effectiveness results for collections distributed over both a local area network and a wide area network. We conclude that, compared to monolithic systems, distributed information retrieval systems can be fast and effective, but that they are not efficient.

Collection selection for managed distributed document databases

“…In principle, particularly in the absence of cooperation, it would be possible for a collection to contain many relevant documents, be highly ranked, but for the collection's ranking mechanism to be unable to find the documents. In practice, in cooperative systems in which information such as term statistics are shared, this is no more likely than in a monolithic system (de Kretser et al, 1998). Thus it is reasonable to compare systems by their ability to find collections with relevant documents, while noting that there remains the issue of how to find documents within collections and combine these results.…”

A pipelined architecture for distributed text query evaluation

“…A straightforward way of distributing the retrieval task is to allocate each computer, or server, a defined fraction of the documents and then build an index for each local document set (Harman et al, 1991;de Kretser et al, 1998;Cahoon et al, 2000). Each index consists of a complete vocabulary for the documents on that computer and, for each term in the vocabulary, an inverted list recording the documents containing the term and (if phrase querying is to be supported) the positions in each document at which the term occurs.…”

A timed mobile agent planning approach for distributed information retrieval in dynamic network environments

“…Tmap1 estimates the agents' routing times by adding the costs of the next adjacent set of nodes in the sorted list. Exceeding the upper bound of the time constraint (the threshold ''T end ''), i.e., the impossibility of performing the task at the node within the time window, is not allowed when assigning an adjacent node to the tour (lines 22,23,26). In this manner, the path of the agents can be decided using the sorted list.…”

“…In this application, information is spread over several nodes, which are commonly geographically separated [22]. Mobile agents migrate to the nodes where the data are located to perform their retrieval tasks there instead of transmitting data across the network.…”