Confusionset-guided Pointer Networks for Chinese Spelling Check

Abstract

This paper proposes Confusionset-guided Pointer Networks for Chinese Spell Check (CSC) task. More concretely, our approach utilizes the off-the-shelf confusionset for guiding the character generation. To this end, our novel Seq2Seq model jointly learns to copy a correct character from an input sentence through a pointer network, or generate a character from the confusionset rather than the entire vocabulary. We conduct experiments on three human-annotated datasets, and results demonstrate that our proposed generative model outperforms all competitor models by a large margin of up to 20% F1 score, achieving state-of-the-art performance on three datasets.

Spelling Error Correction with Soft-Masked BERT

“…Hybrid uses a BiLSTM-based model trained on a generated dataset (Wang et al, 2018b). Confusionset is a Seq2Seq model consisting of a pointer network and copy mechanism (Wang et al, 2019). FASPell adopts a Seq2Seq model for CSC employing BERT as a denoising autoencoder and a decoder (Hong et al, 2019).…”

“…Chinese spelling error correction (CSC) is a special case, but is more challenging due to its conflation with Chinese word segmentation, which received a considerable number of investigations Yu and Li, 2014;Tseng et al, 2015;Wang et al, 2019). Early work in CSC followed the pipeline of error detection, candidate generation, and final candidate selection.…”

“…Some researchers employed unsupervised methods using language models and rules (Yu and Li, 2014;Tseng et al, 2015) and the others viewed it as a sequential labeling problem and employed conditional random fields or hidden Markov models (Tseng et al, 2015;Zhang et al, 2015). Recently, deep learning was applied to spelling error correction (Guo et al, 2019;Wang et al, 2019), and for example, a Seq2Seq model with BERT as encoder was employed (Hong et al, 2019), which transforms the input sentence into a new sentence with spelling errors corrected.…”

“…Spelling error correction is an important task which aims to correct spelling errors in a text either at word-level or at character-level (Yu and Li, 2014;Zhang et al, 2015;Wang et al, 2018b;Hong et al, 2019;Wang et al, 2019). It is crucial for many natural language applications such as search (Martins and Silva, 2004;Gao et al, 2010), optical character recognition (OCR) (Afli et al, 2016;Wang et al, 2018b), and essay scoring (Burstein and Chodorow, 1999).…”

“…Zhang et al (2015), for example, proposed a unified framework for CSC consisting of a pipeline of error de-tection, candidate generation, and final candidate selection using traditional machine learning. Wang et al (2019) proposed a Seq2Seq model with copy mechanism which transforms an input sentence into a new sentence with spelling errors corrected.…”