Learning collection fusion strategies

Voorhees, Ellen M.; Gupta, N. K.; Johnson-Laird, Ben

doi:10.1145/215206.215357

Cited by 139 publications

(118 citation statements)

References 7 publications

Supporting

Mentioning

116

Contrasting

Unclassified

Order By: Relevance

“…[2] examined both combining the results of multiple query formulations and combining retrieval results obtained from multiple retrieval systems. Voorhees, et al [20,21] proposed a merging approach in which the number of documents retrieved from a database was based on the estimated usefulness of that database. Those documents were then merged using a probabilistic approach.…”

Section: Distributed Retrieval Database Selection and Results Mergingmentioning

confidence: 99%

The impact of database selection on distributed searching

Powell

French

Callan

et al. 2000

Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

101

View full text Add to dashboard Cite

The proliferation of online information resources increases the importance of effective and efficient distributed searching. Distributed searching is cast in three parts -database selection, query processing, and results merging. In this paper we examine the effect of database selection on retrieval performance. We look at retrieval performance in three different distributed retrieval testbeds and distill some general results. First we find that good database selection can result in better retrieval effectiveness than can be achieved in a centralized database. Second we find that good performance can be achieved when only a few sites are selected and that the performance generally increases as more sites are selected. Finally we find that when database selection is employed, it is not necessary to maintain collection wide information (CWI), e.g. global idf. Local information can be used to achieve superior performance. This means that distributed systems can be engineered with more autonomy and less cooperation. This work suggests that improvements in database selection can lead to broader improvements in retrieval performance, even in centralized (i.e. single database) systems. Given a centralized database and a good selection mechanism, retrieval performance can be improved by decomposing that database conceptually and employing a selection step.

show abstract

Section: Distributed Retrieval Database Selection and Results Mergingmentioning

confidence: 99%

The impact of database selection on distributed searching

Powell

French

Callan

et al. 2000

Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

101

View full text Add to dashboard Cite

show abstract

“…Environments containing many databases, and in which database contents may c hange often, encourage the use of resource descriptions that can be created automatically. Resource descriptions that must be created and updated manually (e.g., Marcus, 1983 Chakravarthy andHaase, 1995) or that are learned from manual relevance judgements (e.g., Voorhees et al, 1995a) might be di cult or expensive to apply in such e n vironments.…”

Section: Resource Descriptionmentioning

confidence: 99%

“…A simple and robust solution is to to represent e a c h database by a description consisting of the words that occur in the database, and their frequencies of occurrence (Gravano et al, 1994Gravano and Garc a-Molina, 1995Callan et al, 1995b or statistics derived from frequencies of occurrence (Voorhees et al, 1995a). We call this type of representation a unigram language model.…”

Section: Resource Descriptionmentioning

confidence: 99%

Distributed Information Retrieval

Callan

The Information Retrieval Series

228

328

View full text Add to dashboard Cite

A m ulti-database model of distributed information retrieval is presented, in which people are assumed to have access to many searchable text databases. In such a n e n vironment, full-text information retrieval consists of discovering database contents, ranking databases by their expected ability to satisfy the query, s e a r c hing a small number of databases, and merging results returned by di erent databases. This paper presents algorithms for each task. It also discusses how to reorganize conventional test collections into multi-database testbeds, and evaluation methodologies for multi-database experiments. A broad and diverse group of experimental results is presented to demonstrate that the algorithms are e ective, e cient, robust, and scalable.

show abstract

“…Other popular source selection techniques include gGIOSS resource ranking algorithm (Gravano and Garcia-Molina 1995) and ReDDE (Si and Callan 2003 The most relevant papers in this stream are the ones applying decision-theoretic approaches. These include work by Fuhr (1999) on a decision-theoretic approach to source selection and by Voorhees (1995) on an approach to select sources and merge results based on historical data. Etzioni et al (1996) also study the optimal sequence in which to query information sources in a sequential query problem where the broker pays each information source in order to query it.…”

Section: Operational Decisions In Dirmentioning

confidence: 99%

Usercentric Operational Decision Making in Distributed Information Retrieval

Hosanagar

2011

Information Systems Research

View full text Add to dashboard Cite

Information specialists in enterprises regularly use distributed information retrieval (DIR) systems that query a large number of information retrieval (IR) systems, merge the retrieved results, and display them to users. There can be considerable heterogeneity in the quality of results returned by different IR servers. Further, because different servers handle collections of different sizes and have different processing and bandwidth capacities, there can be considerable heterogeneity in their response times. The broker in the DIR system has to decide which servers to query, how long to wait for responses, and which retrieved results to display based on the benefits and costs imposed on users. The benefit of querying more servers and waiting longer is the ability to retrieve more documents. The costs may be in the form of access fees charged by IR servers or user's cost associated with waiting for the servers to respond. We formulate the broker's decision problem as a stochastic mixed-integer program and present analytical solutions for the problem. Using data gathered from FedStats-a system that queries IR engines of several U.S. federal agencies-we demonstrate that the technique can significantly increase the utility from DIR systems. Finally, simulations suggest that the technique can be applied to solve the broker's decision problem under more complex decision environments. Electronic copy available at: http://ssrn.com/abstract=926928 User-Centric Operational Decision-Making in Distributed Information RetrievalKartik Hosanagar + AbstractInformation specialists in enterprises regularly use Distributed Information Retrieval (DIR) systems that query a large number of Information Retrieval (IR) systems, merge the retrieved results and display them to users. There can be considerable heterogeneity in the quality of results returned by different IR servers. Further, since different servers handle collections of different sizes, have different processing and bandwidth capacities, there can be considerable heterogeneity in their response times. The broker in the distributed IR system has to decide which servers to query, how long to wait for responses and which retrieved results to display based on the benefits and costs imposed on users. The benefit of querying more servers and waiting longer is the ability to retrieve more documents. The costs may be in the form of access fees charged by IR servers or user's cost associated with waiting for the servers to respond. We formulate the broker's decision problem as a stochastic mixed integer program and present analytical results for the optimal query set and wait time. Using data gathered from Fedstats -a system that queries IR engines of several US federal agencies -we demonstrate that the technique can significantly increase the utility from DIR systems. Finally, we present a simulation-based optimization technique to solve the broker's decision problem under more complex decision environments. The technique is computationally efficient and can be used to ge...

show abstract

Learning collection fusion strategies

Cited by 139 publications

References 7 publications

The impact of database selection on distributed searching

The impact of database selection on distributed searching

Distributed Information Retrieval

Usercentric Operational Decision Making in Distributed Information Retrieval

Contact Info

Product

Resources

About