Deep Learning for Automatic Diacritics Restoration in Romanian

Nuțu, Maria; Lorincz, Beata; Stan, Adriana

doi:10.1109/iccp48234.2019.8959557

Cited by 8 publications

(7 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such a seq2seq approach, with the RNN-based core, was successfully applied to the Turkish language [43], and, with the LSTM-based core, to Vietnamese texts [5,44]. In [45], Romanian authors investigated four different encoder-decoder architectures operating on the character level: one-layer LSTMs, two types of stacked LSTMs, and the CNN-based method (three-layer CNN with the concatenated output of the encoder and decoder, processed with another two-layer CNN), and determined that the CNN-based approach was the most accurate. Moreover, they compared their seq2seq approaches with the classification-based approach.…”

Section: Deep-learning-based Approachesmentioning

confidence: 99%

Correcting Diacritics and Typos with a ByT5 Transformer Model

et al. 2022

View full text Add to dashboard Cite

Due to the fast pace of life and online communications and the prevalence of English and the QWERTY keyboard, people tend to forgo using diacritics, make typographical errors (typos) when typing in other languages. Restoring diacritics and correcting spelling is important for proper language use and the disambiguation of texts for both humans and downstream algorithms. However, both of these problems are typically addressed separately: the state-of-the-art diacritics restoration methods do not tolerate other typos, but classical spellcheckers also cannot deal adequately with all the diacritics missing.In this work, we tackle both problems at once by employing the newly-developed universal ByT5 byte-level seq2seq transformer model that requires no language-specific model structures. For a comparison, we perform diacritics restoration on benchmark datasets of 12 languages, with the addition of Lithuanian. The experimental investigation proves that our approach is able to achieve results (>98%) comparable to the previous state-of-the-art, despite being trained less and on fewer data. Our approach is also able to restore diacritics in words not seen during training with >76% accuracy. Our simultaneous diacritics restoration and typos correction approach reaches >94% alpha-word accuracy on the 13 languages. It has no direct competitors and strongly outperforms classical spell-checking or dictionary-based approaches. We also demonstrate all the accuracies to further improve with more training. Taken together, this shows the great real-world application potential of our suggested methods to more data, languages, and error classes.

show abstract

Section: Deep-learning-based Approachesmentioning

confidence: 99%

Correcting Diacritics and Typos with a ByT5 Transformer Model

et al. 2022

View full text Add to dashboard Cite

show abstract

“…However, when we used random splits for training, validation, and testing (see Section VI), the proposed method produced a better WER (8.43%) and an even better DER (2.13%). In the second comparison approach, we compared the resulting models (see Section VI) with three well-known and publicly available Arabic diacritization systems, namely, MADAMIRA 3 (morphological analysis-based), Farasa 4 (feature engineering-based), and the Belinkov and Glass model 5 (data-based). We based our comparison on three manually diacritized texts from three different genres with various writing styles and sentence structures.…”

Section: Comparisonmentioning

confidence: 99%

“…Diacritics are marks written above or below words or letters in several languages such as Arabic [1], Turkish [2], and Romanian [3]. Arabic texts are usually written without diacritics, and readers can infer the meanings and correct pronunciations of the words from their contexts.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Arabic Diacritization Using Bidirectional Long Short-Term Memory Neural Networks With Conditional Random Fields

et al. 2020

View full text Add to dashboard Cite

Arabic diacritics play a significant role in distinguishing words with the same orthography but different meanings, pronunciations, and syntactic functions. The presence of Arabic diacritics can be useful in many natural language processing applications, such as text-to-speech tasks, machine translation, and partof-speech tagging. This paper discusses the use of bidirectional long short-term memory neural networks with conditional random fields for Arabic diacritization. This approach requires no morphological analyzers, dictionary, or feature engineering, but rather uses a sequence-to-sequence schema. The input is a sequence of characters that constitute the sentence, and the output consists of the corresponding diacritic(s) for each character in that sentence. The performance of the proposed approach was examined using four datasets with different sizes and genres, namely, the King Abdulaziz City for Science and Technology text-to-speech (KACST TTS) dataset, the Holy Quran, Sahih Al-Bukhary, and the Penn Arabic Treebank (ATB). For training, 60% of the sentences were randomly selected from each dataset, 20% were selected for validation, and 20% were selected for testing. The trained models achieved diacritic error rates of 3.41%, 1.34%, 1.57%, and 2.13% and word error rates of 14.46%, 4.92%, 5.65%, and 8.43% on the KACST TTS, Holy Quran, Sahih Al-Bukhary, and ATB datasets, respectively. Comparison of the proposed method with those used in other studies and existing systems revealed that its results are comparable to or better than those of the stateof-the-art methods.

show abstract

“…In [27], an improvement is developed where a morphological and a syntactical analyzer are used to accelerate the predictions. Since Convolutional Neural Networks (CNNs) can run in parallel, they may be an alternative for RNNs when the speed of diacritization is a major issue [28,29]. They usually lead to a loss in the accuracy.…”

Section: Introductionmentioning

confidence: 99%

Recent Advances in Arabic Syntactic Diacritics Restoration

Hifny

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Restoring Arabic syntactic diacritics based on Long Short-Term Memory (LSTM) networks leads to state-of-the-art performance. These LSTM networks are commonly augmented with Maximum Entropy (MaxEnt) sparse direct connections between the input and the output layers of the tagger. One way to improve such tagger performance is to use an ensemble of taggers. However, an ensemble of taggers may require huge computational and memory resources. In this paper, we implement a knowledge distillation technique where an ensemble of teachers/taggers is used to train a single student tagger. On the other hand, Arabic is a morphologically rich language and has a high Out-Of-Vocabulary (OOV) rate. In addition to word embeddings, we propose to use character embeddings encoded using LSTMs for each word to overcome this problem. On the Arabic tree bank task, our hybrid LSTM/MaxEnt tagger achieves 1.0% absolute WER improvement over a strong baseline using the proposed two techniques.

show abstract

Deep Learning for Automatic Diacritics Restoration in Romanian

Cited by 8 publications

References 3 publications

Correcting Diacritics and Typos with a ByT5 Transformer Model

Correcting Diacritics and Typos with a ByT5 Transformer Model

Arabic Diacritization Using Bidirectional Long Short-Term Memory Neural Networks With Conditional Random Fields

Recent Advances in Arabic Syntactic Diacritics Restoration

Contact Info

Product

Resources

About