Arabic Diacritic Recovery Using a Feature-rich biLSTM Model

Abstract

Diacritics (short vowels) are typically omitted when writing Arabic text, and readers have to reintroduce them to correctly pronounce words. There are two types of Arabic diacritics: The first are core-word diacritics (CW), which specify the lexical selection, and the second are case endings (CE), which typically appear at the end of word stems and generally specify their syntactic roles. Recovering CEs is relatively harder than recovering core-word diacritics due to inter-word dependencies, which are often distant. In this article, we use feature-rich recurrent neural network model that use a variety of linguistic and surface-level features to recover both core word diacritics and case endings. Our model surpasses all previous state-of-the-art systems with a CW error rate (CWER) of 2.9% and a CE error rate (CEER) of 3.7% for Modern Standard Arabic (MSA) and CWER of 2.2% and CEER of 2.5% for Classical Arabic (CA). When combining diacritized word cores with case endings, the resultant word error rates are 6.0% and 4.3% for MSA and CA, respectively. This highlights the effectiveness of feature engineering for such deep neural models.

Start-Before-End and End-to-End: Neural Speech Translation by AppTek and RWTH Aachen University

“…Recurrent long short-term memory (LSTM) networks [12] have been proven to be suitable tools for learning the task entirely from data without using manually designed features [13,14,15]. Their combination with conditional random field (CRF) and the extension to sequence-to-sequence modeling [16] help the model performance [17,18].…”

Arabic Syntactic Diacritics Restoration Using BERT Models

“…The accuracy of the model is measured as the percentage of correct predictions of the Arabic syntactic diacritics (i.e., the last character of the stem of each word). We evaluated the trained models using the complement of the accuracy which is called the case-ending error rate (CEER) [ 3 ].…”

“…Darwish et al [ 3 ] combined a set of character-level features such as stem and part-of-speech (POS) tags with an embedding layer and a bidirectional LSTM. The proposed system restores core word and case ending diacritization.…”

“…In addition, the restoration of Arabic syntactic diacritics (i.e., the last character of the stem of each word) is more challenging than morphological diacritics. Although most of the state-of-the-art systems have a low error rate in the case of morphological diacritics, they still have a high error rate when restoring syntactic diacritics [ 3 ].…”

“…It was found empirically that diacritization errors are often related to the prediction of the syntactic diacritic task when a machine learning approach is developed [3][4][5]. Diacritics restoration is a vital component in Arabic textto-speech (TTS) systems where the phonetic transcription is extremely a ected by diacritization errors [6,7].…”