Fusion of effective retrieval strategies in the same information retrieval system

Abstract

Prior efforts have shown that under certain situations retrieval effectiveness may be improved via the use of data fusion techniques. Although these improvements have been observed from the fusion of result sets from several distinct information retrieval systems, it has often been thought that fusing different document retrieval strategies in a single information retrieval system will lead to similar improvements. In this study, we show that this is not the case. We hold constant systemic differences such as parsing, stemming, phrase processing, and relevance feedback, and fuse result sets generated from highly effective retrieval strategies in the same information retrieval system. From this, we show that data fusion of highly effective retrieval strategies alone shows little or no improvement in retrieval effectiveness. Furthermore, we present a detailed analysis of the performance of modern data fusion approaches, and demonstrate the reasons why they do not perform well when applied to this problem. Detailed results and analyses are included to support our conclusions.

Enhancing the Effectiveness of Ligand‐Based Virtual Screening Using Data Fusion

“…From a combinatorial study of all of the 31 combinations possible with five different scoring functions, Yang et al concluded that the best results were obtained with just two of the functions that performed well on their own and that exhibited different rank-score graphs; they also noted that the fusion of ranks performed at least as well as the fusion of scores. These findings are analogous to previous reports in the IR literature: for example, Ng and Kantor have suggested the use of the Kendall rank correlation coefficient to determine the degree of correlation between the rankings from two scoring functions, and hence the extent to which it would be worth including them in a consensus [83] (see also Hsu and Tahesi [55] and Beitzel et al [85]). Yang et al suggest that the rank-score graph could be used to select effective combinations of scoring functions in the absence of actual performance data.…”

“…This has been the subject of debate in the IR and pattern recognition context for some years [31,55,[82][83][84][85] but the first such report in the chemoinformatics area was a simulation study by Wang and Wang [86]. Each of a set of 5000 hypothetical compounds was assigned a random number representing its experimental binding affinity from a Normal distribution; the 100 molecules with the largest assigned affinities were deemed to be the active molecules for the simulation.…”

Combining Indexing Methods and Query Sizes in Information Retrieval in French

“…The CombMNZ function (Fox and Shaw, 1994) is widely used in data fusion studies (Beitzel et al, 2004). Formula (1) indicates how the CombMNZ function calculates the score of a document J after fusion.…”

Properties of optimally weighted data fusion in CBMIR

“…The data fusion hypothesis of Lee was critically examined by both McCabe et al [11] and Beitzel et al [1]. Both conducted approaches where various system parameters were held constant whilst varying one aspect, such as the ranking model, stemming, stopping, relevance feedback etc.…”

“…Beitzel et al [1] like McCabe also conducted experiments where system parameters are held constant to measure the impact of combination of different aspects of retrieval systems. The work of Beitzel et al specifically examined the combination of "highly effective retrieval strategies".…”