Abstract.To evaluate Information Retrieval Systems on their effectiveness, evaluation programs such as TREC offer a rigorous methodology as well as benchmark collections. Whatever the evaluation collection used, effectiveness is generally considered globally, averaging the results over a set of information needs. As a result, the variability of system performance is hidden as the similarities and differences from one system to another are averaged. Moreover, the topics on which a given system succeeds or fails are left unknown. In this paper we propose an approach based on data analysis methods (correspondence analysis and clustering) to discover correlations between systems and to find trends in topic/system correlations. We show that it is possible to cluster topics and systems according to system performance on these topics, some system clusters being better on some topics. Finally, we propose a new method to consider complementary systems as based on their performances which can be applied for example in the case of repeated queries. We consider the system profile based on the similarity of the set of TREC topics on which systems achieve similar levels of performance. We show that this method is effective when using the TREC ad hoc collection.

Science monitoring is a core issue in the new world of business and research.Companies and institutes need to monitor the activities of its competitors, get information on the market, technologies, or government actions. This paper presents the Tétralogie platform that aims at allowing a user to interactively discover trends in scientific research and communities from large textual collections that provide geographical location. Tétralogie consists of several agents that communicate on users' demand in order to deliver results to them. Meta-data and document content are extracted before being mined. Results are displayed under the form of histograms, networks and geographical maps; these complementary types of presentations increase the possibilities of analysis compared to the use of these tools separately. We illustrate the overall process through a case study of scientific literature analysis and show how the different agents can be combined to discover the structure of a domain. The system predicts correctly the country contribution in future years and delivers the relationships between countries.

No abstract