Although Pseudo-Relevance Feedback (PRF) is a widely used technique for enhancing average retrieval performance, it may actually hurt performance for around one-third of a given set of topics. To enhance the reliability of PRF, Flexible PRF has been proposed, which adjusts the number of pseudo-relevant documents and/or the number of expansion terms for each topic. This paper explores a new, inexpensive Flexible PRF method, called Selective Sampling, which is unique in that it can skip documents in the initial ranked output to look for more "novel" pseudo-relevant documents. While Selective Sampling is only comparable to Traditional PRF in terms of average performance and reliability, per-topic analyses show that Selective Sampling outperforms Traditional PRF almost as often as Traditional PRF outperforms Selective Sampling. Thus, treating the top P documents as relevant is often not the best strategy. However, predicting when Selective Sampling outperforms Traditional PRF appears to be as difficult as predicting when a PRF method fails. For example, our per-topic analyses show that even the proportion of truly relevant documents in the pseudo-relevant set is not necessarily a good performance predictor.

No abstract

No abstract

This paper describes two new features of the BRIDJE system for cross-language information access. The first feature is the partial disambiguation function of the Bi-directional Retriever, which can be used for search request translation in cross-language IR. Its advantage over a "black-box" machine translation approach is consistent across five test collections and across two language permutations: English-Japanese and Japanese-English. The second new feature is the Information Distiller, which performs interactive summarisation of retrieved documents based on Semantic Role Analysis. Our examples illustrate the usefulness of this feature, and our evaluation results show that the precision of Semantic Role Analysis is very high.

No abstract