“They Are Out There, If You Know Where to Look”: Mining Transliterations of OOV Query Terms for Cross-Language Information Retrieval

Udupa, Raghavendra; Saravanan, K.; Bakalov, Anton; Bhole, Abhijit

doi:10.1007/978-3-642-00958-7_39

Cited by 24 publications

(16 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The presence of OOV words across language pairs can impair the estimation quality of CLTRLM because for every target language word whose translation is not found in the dictionary, we fail to get the source language contribution in the generation probability (see Equation 3). To reduce the vocabulary gap we use transliteration of OOV words, since it has been reported that transliteration helps improve the retrieval quality of CLIR (Udupa et al, 2009). Finally to address question d), we evaluate the relative performance of CLTRLM and JCLTRLM.…”

Section: Methodsmentioning

confidence: 99%

“…Proper nouns are important for retrieval (Xu and Croft, 2000), and thus need to be handled appropriately. An intuitive approach is to transliterate English names into Bengali which has proved to be beneficial for Indian language CLIR (Udupa et al, 2009). For transliteration, we applied Google transliterate 6 on the untranslated words of the Bengali queries as obtained by the dictionary-based and the Google translator approaches.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Topical Relevance Model

Ganguly

Leveling

Jones

2012

Information Retrieval Technology

View full text Add to dashboard Cite

Cross-lingual relevance modelling (CLRLM) is a state-of-the-art technique for cross-lingual information retrieval (CLIR) which integrates query term disambiguation and expansion in a unified framework, to directly estimate a model of relevant documents in the target language starting with a query in the source language. However, CLRLM involves integrating a translation model either on the document side if a parallel corpus is available, or on the query side if a bilingual dictionary is available. For low resourced language pairs, large parallel corpora do not exist and the vocabulary coverage of dictionaries is small, as a result of which RLM-based CLIR fails to obtain satisfactory results. Despite the lack of parallel resources for a majority of language pairs, the availability of comparable corpora for many languages has grown considerably in the recent years. Existing CLIR techniques such as cross-lingual relevance models, cannot effectively utilize these comparable corpora, since they do not use information from documents in the source language. We overcome this limitation by using information from retrieved documents in the source language to improve the retrieval quality of the target language documents. More precisely speaking, our model involves a two step approach of first retrieving documents both in the source language and the target language (using query translation), and then improving on the retrieval quality of target language documents by expanding the query with translations of words extracted from the top ranked documents retrieved in the source language which are thematically related (i.e. share the same concept) to the words in the top ranked target language documents. Our key hypothesis is that the query in the source language and its equivalent target language translation retrieve documents which share topics. The ovelapping topics of these top ranked documents in both languages are then used to improve the ranking of the target language documents. Since the model relies on the alignment of topics between language pairs, we call it the cross-lingual topical relevance model (CLTRLM). Experimental results show that the CLTRLM significantly outperforms the standard CLRLM by upto 37% on English-Bengali CLIR, achieving mean average precision (MAP) of up to 60.27% of the Bengali monolingual IR MAP.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Topical Relevance Model

Ganguly

Leveling

Jones

2012

Information Retrieval Technology

View full text Add to dashboard Cite

show abstract

“…They are also critical in cross-lingual applications such as Machine Translation (MT) and Cross-language Information Retrieval (CLIR), as it has been shown that system performance correlates positively with the quality of name conversion across languages (Demner-Fushman and Oard 2002, Mandl and Womser-Hacker 2005,Hermjakobet al 2008, Udupa et al 2009). Bilingual dictionaries constitute the traditional source of information for name conversion across languages, however they offer very limited support due to the fact that, in most languages, names are continuously emerging and evolving.…”

Section: Introductionmentioning

confidence: 99%

Proceedings of the Sixth Named Entity Workshop

2016

View full text Add to dashboard Cite

“…NE translation is crucial for effective cross-language information retrieval (CLIR) [2][3][4], and statistical machine translation (SMT) [5][6][7]. However, the translation of name entities is not successful enough because of the complexity and particularity of a name entity's structure.…”

Section: Introductionmentioning

confidence: 99%