A Shared Attention Mechanism for Interpretation of Neural Automatic Post-Editing Systems

Unanue, Iñigo Jauregi; Borzeshi, Ehsan Zare; Piccardi, Massimo

doi:10.18653/v1/w18-2702

Cited by 7 publications

(5 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…At each decoder time step, an attention vector is calculated as a distribution over all encoded tokens, indicating which token the decoder should "pay attention to". To make the attention mechanism work with multiple encoders, we concatenate the hidden states of the n encoders [h 1 , ..., h n ] and apply an attention layer on the result [70].…”

Section: Core Architecturementioning

confidence: 99%

Deep Generation of Coq Lemma Names Using Elaborated Terms

Nie

Palmskog

et al. 2020

Automated Reasoning

View full text Add to dashboard Cite

Coding conventions for naming, spacing, and other essentially stylistic properties are necessary for developers to effectively understand, review, and modify source code in large software projects. Consistent conventions in verification projects based on proof assistants, such as Coq, increase in importance as projects grow in size and scope. While conventions can be documented and enforced manually at high cost, emerging approaches automatically learn and suggest idiomatic names in Java-like languages by applying statistical language models on large code corpora. However, due to its powerful language extension facilities and fusion of type checking and computation, Coq is a challenging target for automated learning techniques. We present novel generation models for learning and suggesting lemma names for Coq projects. Our models, based on multi-input neural networks, are the first to leverage syntactic and semantic information from Coq's lexer (tokens in lemma statements), parser (syntax trees), and kernel (elaborated terms) for naming; the key insight is that learning from elaborated terms can substantially boost model performance. We implemented our models in a toolchain, dubbed Roosterize, and applied it on a large corpus of code derived from the Mathematical Components family of projects, known for its stringent coding conventions. Our results show that Roosterize substantially outperforms baselines for suggesting lemma names, highlighting the importance of using multi-input models and elaborated terms.

show abstract

Section: Core Architecturementioning

confidence: 99%

Deep Generation of Coq Lemma Names Using Elaborated Terms

Nie

Palmskog

et al. 2020

Automated Reasoning

View full text Add to dashboard Cite

show abstract

“…Most of the papers looked at the application of machine translation, but there is one paper on abstractive summarization (Fan et al, 2018) and one paper on automatic post-editing of translations (Unanue et al, 2018).…”

Section: Summary Of Research Contributionsmentioning

confidence: 99%

“…Model analysis: There were also many methods that analyzed modeling and design decisions, including investigations of individual neuron contributions (Bau et al, 2018), parameter sharing (Jean et al, 2018), controlling output characteristics (Fan et al, 2018), and shared attention (Unanue et al, 2018) 3 Shared Task…”

Section: Summary Of Research Contributionsmentioning

confidence: 99%

Findings of the Second Workshop on Neural Machine Translation and Generation

Birch¹,

Finch

Luong

et al. 2018

Proceedings of the 2nd Workshop on Neural Machine Translation and Generation

View full text Add to dashboard Cite

This document describes the findings of the Second Workshop on Neural Machine Translation and Generation, held in concert with the annual conference of the Association for Computational Linguistics (ACL 2018). First, we summarize the research trends of papers presented in the proceedings, and note that there is particular interest in linguistic structure, domain adaptation, data augmentation, handling inadequate resources, and analysis of models. Second, we describe the results of the workshop's shared task on efficient neural machine translation (NMT), where participants were tasked with creating NMT systems that are both accurate and efficient.

show abstract

“…shown remarkable superiority over their statistical counterparts (Varis and Bojar, 2017;Chatterjee et al, 2017;Junczys-Dowmunt and Grundkiewicz, 2017;Unanue et al, 2018). Most of them cast APE as a multi-source sequence-to-sequence learning problem (Zoph and Knight, 2016): given a source sentence (src) and a machine translation (mt), APE outputs a post-edited translation (pe).…”

Section: Introductionmentioning

confidence: 99%

“…We believe that existing approaches to APE (Varis and Bojar, 2017;Chatterjee et al, 2017;Junczys-Dowmunt and Grundkiewicz, 2017;Unanue et al, 2018) suffer from two major src：I ate a cake yesterday pe ：Ich hatte gestern einen Kuchen gegessen mt ：Ich esse einen Hamburger 1 0 1 0 generate copy interact Figure 1: Learning to copy for APE. Our work is based on two key ideas.…”

Section: Introductionmentioning

confidence: 99%

Learning to Copy for Automatic Post-Editing

Huang

Liu

Luan

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Automatic post-editing (APE), which aims to correct errors in the output of machine translation systems in a post-processing step, is an important task in natural language processing. While recent work has achieved considerable performance gains by using neural networks, how to model the copying mechanism for APE remains a challenge. In this work, we propose a new method for modeling copying for APE. To better identify translation errors, our method learns the representations of source sentences and system outputs in an interactive way. These representations are used to explicitly indicate which words in the system outputs should be copied, which is useful to help CopyNet (Gu et al., 2016) better generate post-edited translations. Experiments on the datasets of the WMT 2016-2017 APE shared tasks show that our approach outperforms all best published results.

show abstract

A Shared Attention Mechanism for Interpretation of Neural Automatic Post-Editing Systems

Cited by 7 publications

References 15 publications

Deep Generation of Coq Lemma Names Using Elaborated Terms

Deep Generation of Coq Lemma Names Using Elaborated Terms

Findings of the Second Workshop on Neural Machine Translation and Generation

Learning to Copy for Automatic Post-Editing

Contact Info

Product

Resources

About