Cross-lingual relevance models

Abstract

We propose a formal model of Cross-Language Information Retrieval that does not rely on either query translation or document translation. Our approach leverages recent advances in language modeling to directly estimate an accurate topic model in the target language, starting with a query in the source language. The model integrates popular techniques of disambiguation and query expansion in a unified formal framework. We describe how the topic model can be estimated with either a parallel corpus or a dictionary. We test the framework by constructing Chinese topic models from English queries and using them in the CLIR task of TREC9. The model achieves performance around 95% of the strong mono-lingual baseline in terms of average precision. In initial precision, our model outperforms the monolingual baseline by 20%. The main contribution of this work is the unified formal model which integrates techniques that are essential for effective Cross-Language Retrieval.

Negative query generation: bridging the gap between query likelihood retrieval models and relevance

“…The query likelihood method and the KL-divergence method have been shown to perform well for a variety of retrieval tasks, including ad-hoc retrieval (Ponte and Croft 1998;Zhai and Lafferty 2001b;Lafferty and Zhai 2001), cross-lingual information retrieval (Xu et al 2001;Lavrenko et al 2002), distributed information retrieval (Xu and Croft 1999;Si et al 2002), structured document retrieval (Ogilvie and Callan 2003), personalized and context-sensitive search (Shen et al 2005;Tan et al 2006), modeling redundancy (Zhang et al 2002), predicting query difficulty (Cronen-Townsend et al 2002), expert finding (Balog et al 2006;Fang and Zhai 2007), passage retrieval (Liu and Croft 2002;Lv and Zhai 2009), subtopic retrieval , etc.…”

“…The language modeling approach to information retrieval (Ponte and Croft 1998) has recently enjoyed much success for many different retrieval tasks (Ponte and Croft 1998;Xu and Croft 1999;Zhai and Lafferty 2001b;Lafferty and Zhai 2001;Xu et al 2001;Lavrenko et al 2002;Si et al 2002;Zhang et al 2002;Cronen-Townsend et al 2002;Liu and Croft 2002;Zhai et al 2003;Ogilvie and Callan 2003;Shen et al 2005;Tan et al 2006;Balog et al 2006;Fang and Zhai 2007;Zhai 2008;Lv and Zhai 2009;Tsagkias et al 2011). In the language modeling approach, we assume that a query is a sample drawn from a language model: given a query Q and a document D, we compute the likelihood of ''generating'' query Q with a document language model estimated based on document D. We can then rank documents based on the likelihood of generating the query, i.e., query likelihood.…”

Research on English-Chinese Cross-Language Information Retrieval

“…So CLIR is more complex than monolingual IR. At present, query translation has become the most popular technique for CLIR [6]. Through query translation, CLIR task can be converted into monolingual IR task.…”

Translating unknown queries with web corpora for cross-language information retrieval

“…Conventionally CLIR approaches [4,7,8,12,21] have focused mainly on incorporating dictionaries and domain-specific bilingual corpora for query translation [6,10,18]. The general assumption of such approaches is that the incorrect translation of a few query terms in a query is tolerable and can be remedied via query expansion in the process of document retrieval.…”

“…For example, previous Web search engine log analyses revealed that the average query length for a Web search was about 2.3 words in English [17] and 3.18 characters in Chinese [13]. Conventional CLIR approaches [8,21] that are based on domainspecific corpora might not be applicable to dealing with the translation of short queries with unknown terms. First, sufficiently large bilingual corpora for certain subject domains and language pairs are not always available.…”