Refined stop-words and morphological variants solutions applied to Hindi-English cross-lingual information retrieval

Sharma, Vijay K; Mittal, Namita

doi:10.3233/jifs-169933

Cited by 8 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, neural machine translation (NMT) has shown their superiority in a variety of translation tasks (Ott et al, 2018;Hassan et al, 2018). Several studies begin to explore the feasibility and improvements of NMT for QT task (Sarwar et al, 2019;Sharma and Mittal, 2019).…”

Section: Introductionmentioning

confidence: 99%

Domain Transfer based Data Augmentation for Neural Query Translation

Liang

Yang

Zhang

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

Query translation (QT) serves as a critical factor in successful cross-lingual information retrieval (CLIR). Due to the lack of parallel query samples, neural-based QT models are usually optimized with synthetic data which are derived from large-scale monolingual queries. Nevertheless, such kind of pseudo corpus is mostly produced by a general-domain translation model, making it be insufficient to guide the learning of QT model. In this paper, we extend the data augmentation with a domain transfer procedure, thus to revise synthetic candidates to search-aware examples. Specifically, the domain transfer model is built upon advanced Transformer, in which layer coordination and mixed attention are exploited to speed up the refining process and leverage parameters from a pre-trained cross-lingual language model. In order to examine the effectiveness of the proposed method, we collected French-to-English and Spanish-to-English QT test sets, each of which consists of 10,000 parallel query pairs with careful manual-checking. Qualitative and quantitative analyses reveal that our model significantly outperforms strong baselines and the related domain transfer methods on both translation quality and retrieval accuracy. 1

show abstract

Section: Introductionmentioning

confidence: 99%

Domain Transfer based Data Augmentation for Neural Query Translation

Liang

Yang

Zhang

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…Recently, neural machine translation (NMT) has shown their superiority in a variety of translation tasks [5,10]. Several studies begin to explore the feasibility and improvements of NMT for QT task [16,19].…”

Section: Introductionmentioning

confidence: 99%

Constraint Translation Candidates: A Bridge between Neural Query Translation and Cross-lingual Information Retrieval

Bi¹,

Liang²,

Yang³

et al. 2020

Preprint

View full text Add to dashboard Cite

Query translation (QT) is a key component in cross-lingual information retrieval system (CLIR). With the help of deep learning, neural machine translation (NMT) has shown promising results on various tasks. However, NMT is generally trained with large-scale out-of-domain data rather than in-domain query translation pairs. Besides, the translation model lacks a mechanism at the inference time to guarantee the generated words to match the search index. The two shortages of QT result in readable texts for human but inadequate candidates for the downstream retrieval task. In this paper, we propose a novel approach to alleviate these problems by limiting the open target vocabulary search space of QT to a set of important words mined from search index database. The constraint translation candidates are employed at both of training and inference time, thus guiding the translation model to learn and generate well performing target queries. The proposed methods are exploited and examined in a real-word CLIR system-Aliexpress e-Commerce search engine. 1 Experimental results demonstrate that our approach yields better performance on both translation quality and retrieval accuracy than the strong NMT baseline.

show abstract

“…The proposed system is based on deep residual networks (ResNets) and its performance is compared compared with the performances of other deep neural network architectures including CNN, cascade CNN-long short-term memory (LSTM) and shallow architecture of artificial neural networks (ANNs). The study in [28] is related to Cross-Lingual Information Retrieval (CLIR), where a user can submit a query in a language different from the target documents languages. The focus of the paper is on proposing a translation induction algorithm, which incorporates the refined stop-words list, morphological variants solutions, and translates the words based on the contextual words.…”

mentioning

confidence: 99%