Capturing document context inside sentence-level neural machine translation models with self-training

Mansimov, Elman; Melis, Gábor; Lei, Yu

doi:10.18653/v1/2021.codi-main.14

Cited by 5 publications

(3 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, some improvements obtained with context-aware models, as measured by standard translation metrics, may be attributed to context-driven regularisation acting as a noise generator, particularly with small-scale data (Kim et al, 2019;Li et al, 2020). Nonetheless, several studies have established that context information can indeed be effectively modelled to tackle discursive phenomena in NMT beyond the sentence level (Liu and Zhang, 2020;Rikters and Nakazawa, 2021;Xu et al, 2021;Mansimov et al, 2021;Gete et al, 2023).…”

Section: Related Workmentioning

confidence: 99%

“…Alternatively, sentence-level translations can be refined via reinforcement learning (Xiong et al, 2019;Mansimov et al, 2021) or monolingual repair to post-edit contextual errors in the target language (Voita et al, 2019a). More recently, the use of pretrained language models has been explored for the task, using them to encode the context (Wu et al, 2022) or to initialize NMT models (Huang et al, 2023).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

TANDO^+: Corpus and Baselines for Document-level Machine Translation in Basque-Spanish and Basque-French

Gete,

Etchegoyhen,

Labaka

et al. 2024

Preprint

View full text Add to dashboard Cite

Context-aware Neural Machine Translation can potentially enhance automated translation quality through effective modelling of context beyond the sentence level. However, suitable corpora for contextual modelling are still scarce, presenting a significant challenge for the training and evaluation of context-aware systems. To address this challenge, we describe \textsc{tando^+}, a document-level corpus for the under-resourced language pairs Basque-French and Basque-Spanish. We provide a detailed description of this corpus, which is to be shared with the scientific community. The corpus comprises parallel data from diverse domains (literature, subtitles, and news) and incorporates context-level information. Additionally, it provides manually crafted contrastive test sets for Basque-Spanish, designed for comprehensive assessment of gender and register contextual phenomena. Additionally, we train and evaluate sentence-level baseline models and several state-of-the-art contextual variants. Our results and analyses indicate that the corpus is well-suited to train and evaluate context-aware machine translation systems for the two selected under-resourced language pairs.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

TANDO^+: Corpus and Baselines for Document-level Machine Translation in Basque-Spanish and Basque-French

Gete,

Etchegoyhen,

Labaka

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…In this section we discuss a few additional less related papers which were not included in our survey in Section A.1. Our discourse paraphraser is closely related to work on contextual machine translation, where source/target context is used to improve sentence-level machine translation (House, 2006;Jean et al, 2017;Wang et al, 2017;Tiedemann and Scherrer, 2017;Kuang et al, 2018;Agrawal et al, 2018;Miculicich et al, 2018;Jean et al, 2019;Voita et al, 2019a;Yin et al, 2021;Mansimov et al, 2021). Prior work has shown that context helps with anaphora resolution (Voita et al, 2018), deixis, ellipsis, and lexical cohesion (Voita et al, 2019b).…”

Section: A2 Other Related Workmentioning

confidence: 99%

Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense

Krishna¹,

Song²,

Karpinska³

et al. 2023

Preprint

View full text Add to dashboard Cite

To detect the deployment of large language models for malicious use cases (e.g., fake content creation or academic plagiarism), several approaches have recently been proposed for identifying AI-generated text via watermarks or statistical irregularities. How robust are these detection algorithms to paraphrases of AI-generated text? To stress test these detectors, we first train an 11B parameter paraphrase generation model (DIPPER) that can paraphrase paragraphs, optionally leveraging surrounding text (e.g., user-written prompts) as context. DIPPER also uses scalar knobs to control the amount of lexical diversity and reordering in the paraphrases. Paraphrasing text generated by three large language models (including GPT3.5-davinci-003) with DIP-PER successfully evades several detectors, including watermarking, GPTZero, DetectGPT, and OpenAI's text classifier. For example, DIP-PER drops the detection accuracy of Detect-GPT from 70.3% to 4.6% (at a constant false positive rate of 1%), without appreciably modifying the input semantics.

show abstract

A survey of context in neural machine translation and its evaluation

Castilho,

Knowles

2024

Nat. lang. processing

View full text Add to dashboard Cite

The question of context in neural machine translation often focuses on topics related to document-level translation or intersentential context. However, there is a wide range of other aspects that can be considered under the umbrella of context. In this work, we survey ways that researchers have incorporated context into neural machine translation systems and the evaluation thereof. This includes building translation systems that operate at the paragraph level or the document level or ones that translate at the sentence level but incorporate information from other sentences. We also consider how issues like terminology consistency, anaphora, and world knowledge or external information can be considered as types of context relevant to the task of machine translation and its evaluation. Closely tied to these topics is the question of how to best evaluate machine translation output in a way that is sensitive to the contexts in which it appears. To this end, we discuss work on incorporating context into both human and automatic evaluations of machine translation quality. Furthermore, we also discuss recent experiments in the field as they relate to the use of large language models in translation and evaluation. We conclude with a view of the future of machine translation, where we expect to see issues of context continue to come to the forefront.

show abstract

Capturing document context inside sentence-level neural machine translation models with self-training

Cited by 5 publications

References 27 publications

TANDO^+: Corpus and Baselines for Document-level Machine Translation in Basque-Spanish and Basque-French

TANDO^+: Corpus and Baselines for Document-level Machine Translation in Basque-Spanish and Basque-French

Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense

A survey of context in neural machine translation and its evaluation

Contact Info

Product

Resources

About