A Comprehensive Survey of Grammatical Error Correction

Wang, Yu; Wang, Yuelin; Dang, Kai; Liu, Jie; Liu, Zhuo

doi:10.1145/3474840

Cited by 28 publications

(18 citation statements)

References 122 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Choe et al (2019) describe a 'sequential transfer learning' approach in which the pretrained model, finetuned on all available annotated data, is finetuned again separately for each test set. A thorough review of the GEC field is made by Wang et al (2020).…”

Section: Related Workmentioning

confidence: 99%

Corpora Generation for Grammatical Error Correction

Lichtarge

Alberti

Kumar

et al. 2019

Proceedings of the 2019 Conference of the North

119

105

View full text Add to dashboard Cite

Grammatical Error Correction (GEC) has been recently modeled using the sequenceto-sequence framework. However, unlike sequence transduction problems such as machine translation, GEC suffers from the lack of plentiful parallel data. We describe two approaches for generating large parallel datasets for GEC using publicly available Wikipedia data.The first method extracts sourcetarget pairs from Wikipedia edit histories with minimal filtration heuristics, while the second method introduces noise into Wikipedia sentences via round-trip translation through bridge languages. Both strategies yield similar sized parallel corpora containing around 4B tokens. We employ an iterative decoding strategy that is tailored to the loosely supervised nature of our constructed corpora. We demonstrate that neural GEC models trained using either type of corpora give similar performance. Fine-tuning these models on the Lang-8 corpus and ensembling allows us to surpass the state of the art on both the CoNLL-2014 benchmark and the JFLEG task. We provide systematic analysis that compares the two approaches to data generation and highlights the effectiveness of ensembling. * * Equal contribution. Listing order is random. Jared conducted systematic experiments to determine useful variants of the Wikipedia revisions corpus, pre-training and finetuning strategies, and iterative decoding. Chris implemented the ensemble and provided background knowledge and resources related to GEC. Shankar ran training and decoding experiments using round-trip translated data. Jared, Chris and Shankar wrote the paper. Noam identified Wikipedia revisions as a source of training data. Noam developed the heuristics for using the full Wikipedia revisions at scale and conducted initial experiments to train Transformer models for GEC. Noam and Niki provided guidance on training Transformer models using the Tensor2Tensor toolkit. Simon proposed using round-trip translations as a source for training data, and corrupting them with common errors extracted from Wikipedia revisions. Simon generated such data for this paper.

show abstract

Section: Related Workmentioning

confidence: 99%

Corpora Generation for Grammatical Error Correction

Lichtarge

Alberti

Kumar

et al. 2019

Proceedings of the 2019 Conference of the North

119

105

View full text Add to dashboard Cite

show abstract

“…sNMT and sEd: Same architectures as NMT and Ed respectively, with addition of the synthetic data after the post-edited data (DE_PE) used for system specialisation. (see similar approach for grammar error correction, Wang et al, 2021)…”

Section: Systemsmentioning

confidence: 99%

Producing Standard German Subtitles for Swiss German TV Content

Gerlach,

Mutal,

Pierrette

2022

Ninth Workshop on Speech and Language Processing for Assistive Technologies (SLPAT-2022)

View full text Add to dashboard Cite

In this study we compare two approaches (neural machine translation and edit-based) and the use of synthetic data for the task of translating normalised Swiss German ASR output into correct written Standard German for subtitles, with a special focus on syntactic divergences. Results suggest that NMT is better suited to this task and that relatively simple rule-based generation of synthetic data could be a valuable approach for cases where little training data is available and transformations are simple.

show abstract

“…Spelling Correction and Grammar Error Correction. Spelling correction [39], [40] and grammar error correction [41], [42] are also used for blocking textual adversarial attacks. However, these methods can only deal with attacks that bring grammatical and spelling errors and cannot identify adversarial examples crafted by word substitution.…”

Section: B Adversarial Defensementioning

confidence: 99%

TREATED:Towards Universal Defense against Textual Adversarial Attacks

Zhu,

Gu,

Wang

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent work shows that deep neural networks are vulnerable to adversarial examples. Much work studies adversarial example generation, while very little work focuses on more critical adversarial defense. Existing adversarial detection methods usually make assumptions about the adversarial example and attack method (e.g., the word frequency of the adversarial example, the perturbation level of the attack method). However, this limits the applicability of the detection method. To this end, we propose TREATED, a universal adversarial detection method that can defend against attacks of various perturbation levels without making any assumptions. TREATED identifies adversarial examples through a set of well-designed reference models. Extensive experiments on three competitive neural networks and two widely used datasets show that our method achieves better detection performance than baselines. We finally conduct ablation studies to verify the effectiveness of our method.

show abstract

A Comprehensive Survey of Grammatical Error Correction

Cited by 28 publications

References 122 publications

Corpora Generation for Grammatical Error Correction

Corpora Generation for Grammatical Error Correction

Producing Standard German Subtitles for Swiss German TV Content

TREATED:Towards Universal Defense against Textual Adversarial Attacks

Contact Info

Product

Resources

About