The impact of database selection on distributed searching

Abstract

The proliferation of online information resources increases the importance of effective and efficient distributed searching. Distributed searching is cast in three parts -database selection, query processing, and results merging. In this paper we examine the effect of database selection on retrieval performance. We look at retrieval performance in three different distributed retrieval testbeds and distill some general results. First we find that good database selection can result in better retrieval effectiveness than can be achieved in a centralized database. Second we find that good performance can be achieved when only a few sites are selected and that the performance generally increases as more sites are selected. Finally we find that when database selection is employed, it is not necessary to maintain collection wide information (CWI), e.g. global idf. Local information can be used to achieve superior performance. This means that distributed systems can be engineered with more autonomy and less cooperation. This work suggests that improvements in database selection can lead to broader improvements in retrieval performance, even in centralized (i.e. single database) systems. Given a centralized database and a good selection mechanism, retrieval performance can be improved by decomposing that database conceptually and employing a selection step.

Context Disambiguation Based Semantic Web Search for Effective Information Retrieval

“…A lot of research in meta search and distributed retrieval investigates mapping user queries to a set of categories or collections (Dolin et al, 1998;Fuhr, 1999;Gauch et al, 1996;Gravano and Garcia-Molina, 1995;Howe and Dreilinger, 1997;Powell et al, 2003;Xu and Croft, 1999;Yu et al, 2001). However, all of the above techniques return the same results for a given query regardless of the submitted query.…”

A Meta-search Method with Clustering and Term Correlation

“…Since doc 11 and doc 12 in cluster c 1 together contribute to sim(A, B), sim(B, C), and sim(A, C), while in c 2 , only doc 21 contributes to the similarity of all the C 2 4 = 6 possible term pairs, we compute the cluster weights to be…”

“…Most of the existing server selection methods [11] keep statistical information about the participant search servers' indexes. They invariably maintain a list of terms found in the indexed documents, while the weight of a term may vary across different methods.…”

Report on CLEF-2001 Experiments: Effective Combined Query-Translation Approach

“…the document score of the retrieved record in the first position). As a variant of this normalized score merging scheme, Powell et al [28] suggest normalizing the document score rsv j according to the following formula: …”