MaskGEC: Improving Neural Grammatical Error Correction via Dynamic Masking

Zhao, Zewei; Wang, Houfeng

doi:10.1609/aaai.v34i01.5476

Cited by 41 publications

(31 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kiyono et al demonstrated that the back-translation approach is effective to enlarge the training data to improve the correction performance, compared with directly introducing the noise to build the training data [41]. Zhao et al proposed a method to improve the grammar error correction model based on neural machine translation through dynamic masking, which solves the model's need for a corpus of "error-correct" sentence pairs [42].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Pre-Training-Based Grammatical Error Correction Model for the Written Language of Chinese Hearing Impaired Students

Chen

Zhang

2022

IEEE Access

View full text Add to dashboard Cite

Grammatical error correction has been considered as an application closely related to daily life and an important shared task in many prestigious competitions and workshops. The neural machine translation with an encoder-decoder architecture containing language models has been the fundamental solution for the grammatical error correction. Whereas Grammatical error correction task on texts of deaf people or its solution has not been seen yet, and common Grammatical error correction tasks are suffering several challenges, such as insufficient training data, insufficient accuracy due to the unsatisfactory capacity of extracting semantic and grammatical patterns. Under these circumstances, we proposed a novel encoderdecoder architecture based on multi-head self-attention along with multiple strategies, which excels at extracting deep representations from the corrupted sentences of deaf students and further reconstructing the sentences into grammatical ones. Via the re-ranking strategy, our model can correct various kinds of errors including spelling and complex syntax errors. The ablation experiments prove that the semantic extracting of self-attention mechanism excluding the position encoding with the word order shuffle operation can significantly learn the deaf students' sentence patterns whose word order is quite different from the ones of hearing people and improve the correction scores. The pre-training can enhance the restoring efficiency of sentence structure in the decoding process. The comparison experiments with baseline models show that our model obtains superior performance either in the deaf students' grammatical error correction or in a common grammatical error correction shared task.

show abstract

Section: Related Workmentioning

confidence: 99%

“…As for comparison on the shared GEC task of NLPCC 2018, the MaxMatch(M²) Scorer [42,51] is introduced to evaluate the performance of competitors. It allows the phrases from the prediction sentence with the maximal overlap with the gold standard to be selected to form the set of prediction edits { , … }.…”

Section: Evaluation Criteriamentioning

confidence: 99%

Pre-Training-Based Grammatical Error Correction Model for the Written Language of Chinese Hearing Impaired Students

Chen

Zhang

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…In the case of China, which is an East Asian cultural region such as Korea, there is a lack of corpus to be used for GEC learning, like Korean. So, in the case of Zhao and Wang [14], the method of giving noise was overcome by applying the dynamic masking technique.…”

Section: Related Workmentioning

confidence: 99%

Korean Grammatical Error Correction Based on Transformer with Copying Mechanisms and Grammatical Noise Implantation Methods

Lee

Shin

Lee

et al. 2021

Sensors

View full text Add to dashboard Cite

Grammatical Error Correction (GEC) is the task of detecting and correcting various grammatical errors in texts. Many previous approaches to the GEC have used various mechanisms including rules, statistics, and their combinations. Recently, the performance of the GEC in English has been drastically enhanced due to the vigorous applications of deep neural networks and pretrained language models. Following the promising results of the English GEC tasks, we apply the Transformer with Copying Mechanism into the Korean GEC task by introducing novel and effective noising methods for constructing Korean GEC datasets. Our comparative experiments showed that the proposed system outperforms two commercial grammar check and other NMT-based models.

show abstract

“…Early studies (Gamon et al, 2008;Tetreault et al, 2010;Dahlmeier and Ng, 2011;Berend et al, 2013;Rozovskaya and Roth, 2014) take GEC as a classification task and rely much on hand-crafted rules. More recently, the techniques of statistical machine translation and neural machine translation are applied to GEC and have made remarkable performance (Behera and Bhattacharyya, 2013;Junczys-Dowmunt and Grundkiewicz, 2016;Junczys-Dowmunt et al, 2018;Chollampatt and Ng, 2018;Zhao et al, 2019;Awasthi et al, 2019;Kiyono et al, 2019;Kaneko et al, 2020;Omelianchuk et al, 2020;Zhao and Wang, 2020).…”

Section: Related Workmentioning

confidence: 99%

Data Augmentation of Incorporating Real Error Patterns and Linguistic Knowledge for Grammatical Error Correction

Xia¹,

He²

2021

Proceedings of the 25th Conference on Computational Natural Language Learning

View full text Add to dashboard Cite

Data augmentation aims at expanding training data with clean text using noising schemes to improve the performance of grammatical error correction (GEC). In practice, there are a great number of real error patterns in the manually annotated training data. We argue that these real error patterns can be introduced into clean text to effectively generate more real and high quality synthetic data, which is not fully explored by previous studies. Moreover, we also find that linguistic knowledge can be incorporated into data augmentation for generating more representative and more diverse synthetic data. In this paper, we propose a novel data augmentation method that fully considers the real error patterns and the linguistic knowledge for the GEC task. We conduct extensive experiments on public data sets and the experimental results show that our method outperforms several strong baselines with far less external unlabeled clean text data, highlighting its extraordinary effectiveness in the GEC task that lacks large-scale labeled training data.

show abstract

MaskGEC: Improving Neural Grammatical Error Correction via Dynamic Masking

Cited by 41 publications

References 14 publications

Pre-Training-Based Grammatical Error Correction Model for the Written Language of Chinese Hearing Impaired Students

Pre-Training-Based Grammatical Error Correction Model for the Written Language of Chinese Hearing Impaired Students

Korean Grammatical Error Correction Based on Transformer with Copying Mechanisms and Grammatical Noise Implantation Methods

Data Augmentation of Incorporating Real Error Patterns and Linguistic Knowledge for Grammatical Error Correction

Contact Info

Product

Resources

About