A Simple but Effective Classification Model for Grammatical Error Correction

Kaili, Zhu; Wang, Chuan; Li, Ruobing; Liu, Yang; Hu, Tianlei; Lin, Hui

doi:10.48550/arxiv.1807.00488

Cited by 3 publications

(1 citation statement)

References 10 publications

(7 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All recent neural approaches are sequential ED models that predict either word sequences (Zhao et al, 2019;Lichtarge et al, 2019) or character sequences (Xie et al, 2016) using either multi-layer RNNs (Ji et al, 2017; or CNNs (Chollampatt and Ng, 2018a;Ge et al, 2018a) or Transformers Lichtarge et al, 2019). Our sequence labeling formulation is similar to (Yannakoudakis et al, 2017) and (Kaili et al, 2018) but the former uses it to only detect errors and the latter only corrects five error-types using separate classifiers. Edits have been exploited in earlier GEC systems too but very unlike our method of re-architecting the core model to label input sequence with edits.…”

Section: Spell Correctionmentioning

confidence: 99%

Parallel Iterative Edit Models for Local Sequence Transduction

Awasthi¹,

Sarawagi²,

Goyal³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

112

View full text Add to dashboard Cite

We present a Parallel Iterative Edit (PIE) model for the problem of local sequence transduction arising in tasks like Grammatical error correction (GEC). Recent approaches are based on the popular encoder-decoder (ED) model for sequence to sequence learning. The ED model auto-regressively captures full dependency among output tokens but is slow due to sequential decoding. The PIE model does parallel decoding, giving up the advantage of modelling full dependency in the output, yet it achieves accuracy competitive with the ED model for four reasons: 1. predicting edits instead of tokens, 2. labeling sequences instead of generating sequences, 3. iteratively refining predictions to capture dependencies, and 4. factorizing logits over edits and their token argument to harness pretrained language models like BERT. Experiments on tasks spanning GEC, OCR correction and spell correction demonstrate that the PIE model is an accurate and significantly faster alternative for local sequence transduction. The code and pre-trained models for GEC are available at https://github. com/awasthiabhijeet/PIE.

show abstract

Section: Spell Correctionmentioning

confidence: 99%

Parallel Iterative Edit Models for Local Sequence Transduction

Awasthi¹,

Sarawagi²,

Goyal³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

112

View full text Add to dashboard Cite

show abstract

Noisy Channel for Low Resource Grammatical Error Correction

Flachs¹,

Lacroix

Søgaard

2019

Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications

View full text Add to dashboard Cite

This paper describes our contribution to the low-resource track of the BEA 2019 shared task on Grammatical Error Correction (GEC). Our approach to GEC builds on the theory of the noisy channel by combining a channel model and language model. We generate confusion sets from the Wikipedia edit history and use the frequencies of edits to estimate the channel model. Additionally, we use two pretrained language models: 1) Google's BERT model, which we fine-tune for specific error types and 2) OpenAI's GPT-2 model, utilizing that it can operate with previous sentences as context. Furthermore, we search for the optimal combinations of corrections using beam search.

show abstract