Encode, Tag, Realize: High-Precision Text Editing

Malmi, Eric; Krause, Sebastian M.; Rothe, Sascha; Mirylenka, Daniil; Severyn, Aliaksei

doi:10.48550/arxiv.1909.01187

Cited by 8 publications

(13 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specifically, LaserTagger (Malmi et al 2019) predicts editions between keeping, deleting or adding a new token/phrase from a handcrafted vocabulary. PIE (Awasthi et al 2019) iteratively predicts token-level editions for a fixed number of iterations in an non-autoregressive way.…”

Section: Gec By Generating Editsmentioning

confidence: 99%

“…For the English GEC task, we compare the proposed S2A model to several representative systems, including three seq2seq baselines (Transformer Big, BERT-fuse (Kaneko et al 2020), PRETLarge (Kiyono et al 2019)), four sequence tagging models (LaserTagger (Malmi et al 2019), PIE (Awasthi et al 2019), GECToR (Omelianchuk et al 2020), Seq2Edits (Stahlberg and Kumar 2020)), and a pipeline model ESD+ESC (Chen et al 2020). Specifically, for GECToR, we report their results when utilizing the pretrained BERT model, XLNet model (Yang et al 2019) and the results that integrate three different pre-trained language models in an ensemble.…”

Section: Baselinesmentioning

confidence: 99%

“…Consequentially, the seq2seq models may suffer from these potential risks in practice which conflict with the second primary goal of the GEC task. Recently, sequence tagging methods (Malmi et al 2019;Awasthi et al 2019;Omelianchuk et al 2020;Stahlberg and Kumar 2020) consider GEC as a text editing task by detecting and applying edits to the original sentences, therefore bypassing the above mentioned problems of seq2seq models. Nevertheless, the edits are usually constrained by human designed or automatically generated lexical rules (Omelianchuk et al 2020;Stahlberg and Kumar 2020) and vocabularies (Awasthi et al 2019;Malmi et al 2019), which limits the generality and transferability of these methods.…”

Section: Introductionmentioning

confidence: 99%

“…Recently, sequence tagging methods (Malmi et al 2019;Awasthi et al 2019;Omelianchuk et al 2020;Stahlberg and Kumar 2020) consider GEC as a text editing task by detecting and applying edits to the original sentences, therefore bypassing the above mentioned problems of seq2seq models. Nevertheless, the edits are usually constrained by human designed or automatically generated lexical rules (Omelianchuk et al 2020;Stahlberg and Kumar 2020) and vocabularies (Awasthi et al 2019;Malmi et al 2019), which limits the generality and transferability of these methods. Moreover, when it comes to corrections which need longer insertions, most of these sequence tagging methods rely on iterative corrections, which can reduce the fluency.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Sequence-to-Action: Grammatical Error Correction with Action Guided Sequence Generation

Guo

Zhang

et al. 2022

AAAI

View full text Add to dashboard Cite

The task of Grammatical Error Correction (GEC) has received remarkable attention with wide applications in Natural Language Processing (NLP) in recent years. While one of the key principles of GEC is to keep the correct parts unchanged and avoid over-correction, previous sequence-to-sequence (seq2seq) models generate results from scratch, which are not guaranteed to follow the original sentence structure and may suffer from the over-correction problem. In the meantime, the recently proposed sequence tagging models can overcome the over-correction problem by only generating edit operations, but are conditioned on human designed language-specific tagging labels. In this paper, we combine the pros and alleviate the cons of both models by proposing a novel Sequence-to-Action (S2A) module. The S2A module jointly takes the source and target sentences as input, and is able to automatically generate a token-level action sequence before predicting each token, where each action is generated from three choices named SKIP, COPY and GENerate. Then the actions are fused with the basic seq2seq framework to provide final predictions. We conduct experiments on the benchmark datasets of both English and Chinese GEC tasks. Our model consistently outperforms the seq2seq baselines, while being able to significantly alleviate the over-correction problem as well as holding better generality and diversity in the generation results compared to the sequence tagging models.

show abstract

Section: Gec By Generating Editsmentioning

confidence: 99%

Section: Baselinesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Sequence-to-Action: Grammatical Error Correction with Action Guided Sequence Generation

Guo

Zhang

et al. 2022

AAAI

View full text Add to dashboard Cite

show abstract

“…These models generate full sequences of code tokens left-to-right with any prefix acting as the (partial) user intent. While LMs generate realistic-looking outputs, they are known to occasionally "hallucinate" [27,22,23,19], i.e. generate plausible but incorrect content.…”

Section: Introductionmentioning

confidence: 99%

Learning to Complete Code with Sketches

Guo¹,

Svyatkovskiy²,

Yin³

et al. 2021

Preprint

View full text Add to dashboard Cite

Traditional generative models are limited to predicting sequences of terminal tokens. However, ambiguities in the generation task may lead to incorrect outputs. Towards addressing this, we introduce GRAMMFORMERs, transformer-based grammarguided models that learn (without explicit supervision) to generate sketches -sequences of tokens with holes. Through reinforcement learning, GRAMMFORMERs learn to introduce holes avoiding the generation of incorrect tokens where there is ambiguity in the target task. We train GRAMMFORMERs for statement-level source code completion, i.e. the generation of code snippets given an ambiguous user intent, such as a partial code context. We evaluate GRAMMFORMERs on code completion for C# and Python and show that it generates 10-50% more accurate sketches compared to traditional generative models and 37-50% longer sketches compared to sketch-generating baselines trained with similar techniques.Preprint. Under review.

show abstract

SARG: A Novel Semi Autoregressive Generator for Multi-turn Incomplete Utterance Restoration

Huang

Zou

et al. 2021

AAAI

View full text Add to dashboard Cite

Dialogue systems in open domain have achieved great success due to the easily obtained single-turn corpus and the development of deep learning, but the multi-turn scenario is still a challenge because of the frequent coreference and information omission. In this paper, we investigate the incomplete utterance restoration which has brought general improvement over multi-turn dialogue systems in recent studies. Meanwhile, inspired by the autoregression for text generation and the sequence labeling for text editing, we propose a novel semi autoregressive generator (SARG) with the high efficiency and flexibility. Moreover, experiments on Restoration-200k show that our proposed model significantly outperforms the state-of-the-art models in terms of quality and inference speed.

show abstract

Encode, Tag, Realize: High-Precision Text Editing

Cited by 8 publications

References 0 publications

Sequence-to-Action: Grammatical Error Correction with Action Guided Sequence Generation

Sequence-to-Action: Grammatical Error Correction with Action Guided Sequence Generation

Learning to Complete Code with Sketches

SARG: A Novel Semi Autoregressive Generator for Multi-turn Incomplete Utterance Restoration

Contact Info

Product

Resources

About