mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset

Bonifacio, Luiz; Vitor, Jeronymo,; Abonizio, Hugo Queiroz; Campiotti, Israel; Fadaee, Marzieh; Lotufo, Roberto de Alencar; Nogueira, Rodrigo

doi:10.48550/arxiv.2108.13897

Cited by 26 publications

(13 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…-Query Translation: BM25 retrieval using translated queries produced by a specific MT model and original documents in the target language. 8 -Reranking: We rerank query translation baseline results using the public mT5 reranker 9 trained on translated MS MARCO in 8 languages [4].…”

Section: Methodsmentioning

confidence: 99%

“…6 Machine Translation. For CLEF languages, we use MS MARCO passage translations 7 from Bonifacio et al [4], and the same MT model to translate queries. For the HC4 languages, we use directional MT models built on top of a transformer base architecture (6-layer encoder/decoder) using Sockeye [8].…”

Section: Methodsmentioning

confidence: 99%

“…With the availability of translations of the widely-used English ad hoc retrieval resource MS MARCO [4], translate-train (training the retrieval model on a translated collection) using large ad hoc retrieval collections becomes feasible. Prior work explored a dense retrieval approach to translate-train, showing effectiveness gains on monolingual non-English retrieval tasks [29].…”

Section: Related Workmentioning

confidence: 99%

“…Since our focus here is using English queries to retrieve content in non-English languages, we pair the original English queries with machine translations of relevant and non-relevant MS MARCO passages to form new triples. 4 We then train ColBERT-X on these newly constructed triples in the same manner as Col-BERT.…”

Section: Clir Training Strategiesmentioning

confidence: 99%

See 3 more Smart Citations

Transfer Learning Approaches for Building Cross-Language Dense Retrieval Models

Nair¹,

Yang²,

Lawrie³

et al. 2022

Preprint

View full text Add to dashboard Cite

The advent of transformer-based models such as BERT has led to the rise of neural ranking models. These models have improved the effectiveness of retrieval systems well beyond that of lexical term matching models such as BM25. While monolingual retrieval tasks have benefited from large-scale training collections such as MS MARCO and advances in neural architectures, cross-language retrieval tasks have fallen behind these advancements. This paper introduces ColBERT-X, a generalization of the ColBERT multi-representation dense retrieval model that uses the XLM-RoBERTa (XLM-R) encoder to support cross-language information retrieval (CLIR). ColBERT-X can be trained in two ways. In zero-shot training, the system is trained on the English MS MARCO collection, relying on the XLM-R encoder for cross-language mappings. In translate-train, the system is trained on the MS MARCO English queries coupled with machine translations of the associated MS MARCO passages. Results on ad hoc document ranking tasks in several languages demonstrate substantial and statistically significant improvements of these trained dense retrieval models over traditional lexical CLIR baselines.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Clir Training Strategiesmentioning

confidence: 99%

See 2 more Smart Citations

Transfer Learning Approaches for Building Cross-Language Dense Retrieval Models

Nair¹,

Yang²,

Lawrie³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, automatic translation of the GLUE benchmark, Stanford Natural Language Infer-ence (SNLI) Corpus, and SciTail Dataset in Portuguese is provided by [GOMES 2020]. [Bonifacio et al 2021] provides a multilingual translation of the MS MARCO passage ranking dataset.…”

Section: Related Workmentioning

confidence: 99%

Large-scale Translation to Enable Response Selection in Low Resource Languages: A COVID-19 Chatbot Experiment

Aguiar

Cruz²,

Silva³

et al. 2022

Anais Do XXXVII Simpósio Brasileiro De Banco De Dados (SBBD 2022)

View full text Add to dashboard Cite

Natural Language Processing for Low Resource Languages is challenging. The lack of large-scale datasets affects the performance of data-hungry algorithms. To overcome this, we employ data augmentation to enlarge the training data for the task of response selection in multi-turn retrieval-based chatbots. We automatically translated a large-scale English dataset to Brazilian Portuguese (PT_BR) and used it to train a deep neural network. For a COVID-19 chatbot system, our results show that the combination of training with the translated dataset followed by a fine-tuning with the context-specific dataset provides the best results in terms of recall for all studied models. In addition, we make available the translated large-scale PT_BR dataset.

show abstract

Information Retrieval

2024

View full text Add to dashboard Cite

mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset

Cited by 26 publications

References 0 publications

Transfer Learning Approaches for Building Cross-Language Dense Retrieval Models

Transfer Learning Approaches for Building Cross-Language Dense Retrieval Models

Large-scale Translation to Enable Response Selection in Low Resource Languages: A COVID-19 Chatbot Experiment

Information Retrieval

Contact Info

Product

Resources

About