Reducing Word Omission Errors in Neural Machine Translation: A Contrastive Learning Approach

Yang, Zonghan; Cheng, Yong; Liu, Yang; Sun, Maosong

doi:10.18653/v1/p19-1623

Cited by 56 publications

(35 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent work showed that contrastive learning can boost the performance of self-supervised and semi-supervised learning in computer vision tasks (He et al, 2020;Chen et al, 2020a;Khosla et al, 2020). In natural language processing, contrastive learning has been investigated for several tasks, including language modeling (Huang et al, 2018), unsupervised word alignment (Liu and Sun, 2015) and machine translation (Yang et al, 2019;Lee et al, 2020). In this work, we are interested in applying contrastive learning to chest X-ray report generation in a multi-modality setting.…”

Section: Contrastivementioning

confidence: 99%

Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation

Yan

et al. 2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

Radiology report generation aims at generating descriptive text from radiology images automatically, which may present an opportunity to improve radiology reporting and interpretation. A typical setting consists of training encoder-decoder models on image-report pairs with a cross entropy loss, which struggles to generate informative sentences for clinical diagnoses since normal findings dominate the datasets. To tackle this challenge and encourage more clinically-accurate text outputs, we propose a novel weakly supervised contrastive loss for medical report generation. Experimental results demonstrate that our method benefits from contrasting target reports with incorrect but semantically-close ones. It outperforms previous work on both clinical correctness and text generation metrics for two public benchmarks.

show abstract

Section: Contrastivementioning

confidence: 99%

Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation

Yan

et al. 2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

show abstract

“…Compared to the above methods, our approach does not need to generate extra positive samples. Although (Yang et al, 2019) propose a sentence-level margin loss-based method for machine translation to reduce the word omission errors and do not need positive samples too, their negative samples are generated by word omission at the token level and cannot be used in GEC. In contrast, our approach uses beam search to generate erroneous sentences as negative samples at the sentence level, which effectively prevents the model from making mistakes and thus is more suitable for the GEC task.…”

Section: Contrastive Learningmentioning

confidence: 99%

Grammatical Error Correction with Contrastive Learning in Low Error Density Domains

Cao¹,

Yang²,

Ng³

2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

Although grammatical error correction (GEC) has achieved good performance on texts written by learners of English as a second language, performance on low error density domains where texts are written by English speakers of varying levels of proficiency can still be improved. In this paper, we propose a contrastive learning approach to encourage the GEC model to assign a higher probability to a correct sentence while reducing the probability of incorrect sentences that the model tends to generate, so as to improve the accuracy of the model. Experimental results show that our approach significantly improves the performance of GEC models in low error density domains, when evaluated on the benchmark CWEB dataset.

show abstract

“…How to construct examples is an important issue in contrastive learning. For the translation task, Yang changed the number of omitted words, word frequency, and part of speech according to the actual translation, designed different types of negative examples to realize data augmentation [29]. Wu and Meng proposed to use word deletion, reordering, and substitution to achieve it [30,31].…”

Section: Introductionmentioning

confidence: 99%

Deep Contrast Learning Approach for Address Semantic Matching

Chen

She

et al. 2021

Applied Sciences

View full text Add to dashboard Cite

Address is a structured description used to identify a specific place or point of interest, and it provides an effective way to locate people or objects. The standardization of Chinese place name and address occupies an important position in the construction of a smart city. Traditional address specification technology often adopts methods based on text similarity or rule bases, which cannot handle complex, missing, and redundant address information well. This paper transforms the task of address standardization into calculating the similarity of address pairs, and proposes a contrast learning address matching model based on the attention-Bi-LSTM-CNN network (ABLC). First of all, ABLC use the Trie syntax tree algorithm to extract Chinese address elements. Next, based on the basic idea of contrast learning, a hybrid neural network is applied to learn the semantic information in the address. Finally, Manhattan distance is calculated as the similarity of the two addresses. Experiments on the self-constructed dataset with data augmentation demonstrate that the proposed model has better stability and performance compared with other baselines.

show abstract

Reducing Word Omission Errors in Neural Machine Translation: A Contrastive Learning Approach

Cited by 56 publications

References 19 publications

Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation

Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation

Grammatical Error Correction with Contrastive Learning in Low Error Density Domains

Deep Contrast Learning Approach for Address Semantic Matching

Contact Info

Product

Resources

About