Error Detection for Arabic Text Using Neural Sequence Labeling

Abstract

The English language has, thus far, received the most attention in research concerning automatic grammar error correction and detection. However, these tasks have been less investigated for other languages. In this paper, we present the first experiments using neural network models for the task of error detection for Modern Standard Arabic (MSA) text. We investigate several neural network architectures and report the evaluation results acquired by applying cross-validation on the data. All experiments involve a corpus we created and augmented. The corpus has 494 sentences and 620 sentences after augmentation. Our models achieved a maximum precision of 78.09%, recall of 83.95%, and F0.5 score of 79.62% in the error detection task using SimpleRNN. Using an LSTM, we achieved a maximum precision of 79.21%, recall of 93.8%, and F0.5 score of 79.16%. Finally, the best results were achieved using a BiLSTM with a maximum precision of 80.74%, recall of 85.73%, and F0.5 score of 81.55%. We compared the results of the three models to a baseline, which is a commercially available Arabic grammar checker (Microsoft Word 2007). LSTM, BiLSTM, and SimpleRNN all outperformed the baseline in precision and F0.5. Our work shows preliminary results, demonstrating that neural network architectures for error detection through sequence labeling can successfully be applied to Arabic text.

Automatic Detection of Grammatical Errors in English Verbs Based on RNN Algorithm: Auxiliary Objectives for Neural Error Detection Models

“…In formula (17), the value of C α is the number of words whose occurrence times are lower than the set threshold, so by formula (17), the probability is evenly distributed among all rare verb words. In addition, in the experiment, the initial word input is embedded by word set embedding and kept updated during the training process.…”

“…For example, modern research put forward the GED system for verb forms [16]. In the research of using machine learning to realize GED, in order to improve the reliability and accuracy, different neural networks and deep learning algorithms are gradually proposed [17,18]. Some researchers first proposed a neural GED method based on bidirectional long-term and short-term memory neural network (BI LSTM) [19] and then extended and improved it [20].…”

Arabic Machine Translation: A Survey With Challenges and Future Directions

“…While automatic metrics such as BLEU capture the average case for how well a MT model translates sentences, they do not give insight into which linguistic aspects MT models struggle with producing fluent output. Some research investigated MT samples with native speakers so they could review the linguistic aspects of MT errors [13], [40] other research works used neural networks to detect errors [41] or to correct them [42]. However, as the quality of the MT output improves over the time, MT evaluation becomes fully integrated in many frameworks such as Multidimensional Quality Metric (MQM) 10 , Dynamic Quality Evaluation Mode (DQM) 11 and Natural Language Generation (NLG) 12 .…”

“…Some research investigated MT samples with native speakers so they could review the linguistic aspects of MT errors [13], [40]; other research works used neural networks to detect errors [41], [73] or to correct them [42].…”

“… Developing post-editing rules can help in handling many problems such as ordering errors, yet ambiguity remains a challenging problem [41], [80]. For instance, idiomatic expressions are hard to handle using current techniques because they are unable to differentiate the expression when it is intended for its literal or idiomatic meaning.…”