CLEF-IP 2009: Retrieval Experiments in the Intellectual Property Domain

Abstract

The Clef Ip track ran for the rst time within Clef 2009. The purpose of the track was twofold: to encourage and facilitate research in the area of patent retrieval by providing a large clean data set for experimentation; to create a large test collection of patents in the three main European languages for the evaluation of cross lingual information access. The track focused on the task of prior art search. The 15 European teams who participated in the track deployed a rich range of Information Retrieval techniques adapting them to this new speci c domain and task. A large-scale test collection for evaluation purposes was created by exploiting patent citations.

Patent Retrieval

“…Work in this domain has been encouraged through evaluation campaigns, first through the NII 10 Test Collection for IR Systems (NTCIR) [57,60,93,145,146], and later through the Text Retrieval Conference (TREC) (for the chemical domain [129]) and continuing at the moment of writing this review, through the Cross-Language Evaluation Forum (CLEF) [156,157,158,161].…”

Effects of Language and Topic Size in Patent IR: An Empirical Study

“…The selection of topics was done depending on the number of patent citations and the source of these citations (applicant, examiner, etc., sources being recorded in the patents' search report documents and available in the data collections). The extraction of the relevance judgements for these topics was done using the citations, as described in [18], [11]. Characteristic to the patent data is that the number of citations 2 of any given patent is very small compared to the number of relevant documents for a topic in an ad-hoc evaluation campaign.…”

“…The patents are obtained from the Epo and contain text in German, English, and French [18]. The documents are assigned a 'document language', but parts of their content may additionally occur in one or both of the other languages.…”

“…The topics were syntetic documents, created such that at least the claims were available in all 3 languages in the collection [18]. In 2010 the number of topics was reduced to 2000, the topic documents were patent application documents with the claims usually present in one language only [16].…”

“…The effect of this is that in 2009 more patents were returned as relevant compared to the 2010 results when looking at patents and not patent documents 3 . The second reason behind empty bins is to be found in differences in the origins of the topic document for the two campaign years [18], [16]. The 2009 topic files were artificially created to have a large amount of replicated content in three languages, which made the cross-lingual retrieval problem easier by using monolingual searches.…”