Grammatical Error Correction as GAN-like Sequence Labeling

Abstract

In Grammatical Error Correction (GEC), sequence labeling models enjoy fast inference compared to sequence-to-sequence models; however, inference in sequence labeling GEC models is an iterative process, as sentences are passed to the model for multiple rounds of correction, which exposes the model to sentences with progressively fewer errors at each round. Traditional GEC models learn from sentences with fixed error rates. Coupling this with the iterative correction process causes a mismatch between training and inference that affects final performance. In order to address this mismatch, we propose a GAN-like sequence labeling model, which consists of a grammatical error detector as a discriminator and a grammatical error labeler with Gumbel-Softmax sampling as a generator. By sampling from real error distributions, our errors are more genuine compared to traditional synthesized GEC errors, thus alleviating the aforementioned mismatch and allowing for better training. Our results on several evaluation benchmarks demonstrate that our proposed approach is effective and improves the previous state-of-the-art baseline.

Seeking Common but Distinguishing Difference, A Joint Aspect-based Sentiment Analysis Model

“…Recently, NLP has been developed rapidly Li et al, 2019b;Jiang et al, 2020;, and the process is further by deep neural networks (Parnow et al, 2021;Li et al, 2021a) and pre-trained language models (Li et al, 2021b;Zhang et al, 2020b). Aspect-based sentiment analysis was proposed by Pontiki et al (2014) and also received lots of attention in recent years.…”

Seeking Common but Distinguishing Difference, A Joint Aspect-based Sentiment Analysis Model

“…Recently, NLP has been developed rapidly Li et al, 2019b;Jiang et al, 2020;, and the process is further by deep neural networks (Parnow et al, 2021;Li et al, 2021a) and pre-trained language models (Li et al, 2021b;Zhang et al, 2020b). Aspect-based sentiment analysis was proposed by Pontiki et al (2014) and also received lots of attention in recent years.…”