Arabic Spelling Correction using Supervised Learning

Hassan, Youssef; Aly, Mohamed; Atiya, Amir F.

doi:10.3115/v1/w14-3615

Cited by 15 publications

(11 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As expected, following the Advanced mode, our three annotators could annotate an average of 618.93 words per hour which is double those annotated in the Basic mode (only 302.14 words). Adding 12 The guidelines are available upon request.…”

Section: Annotation Analysis and Resultsmentioning

confidence: 99%

“…Techniques and tools reported in the literature for supporting the Arabic spelling errors detection and correction task include morphological analysis [12] [16], finite state transducer with edit distance [9] [8], statistical character level transformation [14], N-gram scores [17] [8], conditional random fields [14] [8], and Naïve Base [15]. Similar to systems described in the literature, Arib utilizes language resources such as dictionaries and corpora as well as the application of different techniques to support the task of Arabic spelling error detection and correction.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Proceedings of the Second Workshop on Arabic Natural Language Processing

Ali¹,

Magdy²,

Renals³

2015

View full text Add to dashboard Cite

ii Foreword Assalamu 3alaykum wa nín hǎo! Welcome to the Second Arabic Natural Language Processing Workshop held at ACL 2015 in Beijing, China.A number of Arabic NLP (or Arabic NLP-related) workshops and conferences have taken place, both in the Arab World and in association with international conferences. The Arabic NLP workshop at ACL 2015 follows in the footsteps of these previous efforts to provide a forum for researchers to share and discuss their ongoing work. As in the first Arabic NLP workshop held at EMNLP 2014 in Doha, Qatar, this workshop includes a shared task on Automatic Arabic Error Correction, which was designed in the tradition of high profile NLP shared tasks such as CONLL's grammar/error detection and numerous machine translation campaigns by NIST/WMT/MEDAR, among others.We received 23 main workshop submissions and selected 15 (65%) for presentation in the workshop. Nine papers will be presented orally and six as part of a poster session. The presentation mode is independent of of the ranking of the papers. The papers cover a diverse set of topics from designing orthography conventions and annotation tools to speech recognition and deep learning for sentiment analysis.The shared task was a success with eight teams from six countries participating. The shared task system descriptions (short) papers are included in the proceedings to document the shared task systems, but were not reviewed with the rest of the papers of the main workshop. These papers will be presented as posters. A long paper describing the shared task will be presented orally.The quantity and quality of the contributions to the main workshop, as well as the shared task, are strong indicators that there is a continued need for this kind of dedicated Arabic NLP workshop.We would like to acknowledge all the hard work of the submitting authors and thank the reviewers for their diligent work and for the valuable feedback they provided. We are also thankful to the work of the shared task committee, website committee and the publication co-chairs. It has been an honor to serve as program co-chairs. We hope that the reader of these proceedings will find them stimulating and beneficial. AbstractDifferent names may be popular in different countries. Hence, person names may give a clue to a person's country of origin. Along with other features, mapping names to countries can be helpful in a variety of applications such as country tagging twitter users. This paper describes the collection of Arabic Twitter user names that are either written in Arabic or transliterated into Latin characters along with their stated geographical locations. To classify previously unseen names, we trained naive Bayes and Support Vector Machine (SVM) multi-class classifiers using primarily bag-of-words features. We are able to map Arabic user names to specific Arab countries with 79% accuracy and to specific regions (Gulf, Egypt, Levant, Maghreb, and others) with 94% accuracy. As for transliterated Arabic names, the accuracy per country and per region was 67...

show abstract

Section: Annotation Analysis and Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Proceedings of the Second Workshop on Arabic Natural Language Processing

Ali¹,

Magdy²,

Renals³

2015

View full text Add to dashboard Cite

show abstract

“…Error detection: Techniques used in the literature for detecting Arabic spelling errors are essentially based on two approaches: the language rules (AlShenaifi et al, 2015;Shaalan et al, 2010;Hassan et al, 2014) or a dictionary (Attia et al, 2014;Zerrouki et al, 2014;. For the first technique, detecting whether a word is misspelled or not depends on morphological analyzers.…”

Section: Auto-correction Of Hamozamentioning

confidence: 99%

Proceedings of the Third Arabic Natural Language Processing Workshop

Habash¹,

Diab²,

Darwish³

et al. 2017

View full text Add to dashboard Cite

A number of Arabic NLP (or Arabic NLP-related) workshops and conferences have taken place in the last few years, both in the Arab World and in association with international conferences. The Arabic NLP workshop at EACL 2017 follows in the footsteps of these previous efforts to provide a forum for researchers to share and discuss their ongoing work. This particular workshop is the third in a series, following the First Arabic NLP workshop held at EMNLP 2014 in Doha, Qatar; and the Second Arabic NLP workshop held at ACL 2015 in Beijing, China.We received 47 submissions and selected 22 (47% acceptance rate) for presentation in the workshop. All papers were reviewed by three reviewers on average. The number of submissions is over twice that of the previous workshop in Beijing, which also had a higher acceptance rate (65%). Ten papers will be presented orally and 12 as part of a poster session. The presentation mode is independent of the ranking of the papers. The papers cover a diverse set of topics from Maltese and Arabic dialect processing to models of semantic similarity and credibility analysis, advances in Arabic treebanking, and error annotation for dyslexic texts.The quantity and quality of the contributions to the workshop are strong indicators that there is a continued need for this kind of dedicated Arabic NLP workshop.We would like to acknowledge all the hard work of the submitting authors and thank the reviewers for the valuable feedback they provided. We hope these proceedings will serve as a valuable reference for researchers and practitioners in the field of Arabic NLP and NLP in general.Nizar Habash, General Chair, on behalf of the organizers of the workshop. AbstractThis paper presents a language identification system designed to detect the language of each word, in its context, in a multilingual documents as generated in social media by bilingual/multilingual communities, in our case speakers of Algerian Arabic. We frame the task as a sequence tagging problem and use supervised machine learning with standard methods like HMM and Ngram classification tagging. We also experiment with a lexicon-based method. Combining all the methods in a fall-back mechanism and introducing some linguistic rules, to deal with unseen tokens and ambiguous words, gives an overall accuracy of 93.14%. Finally, we introduced rules for language identification from sequences of recognised words.

show abstract

“…Error detection: Techniques used in the literature for detecting Arabic spelling errors are essentially based on two approaches: the language rules (AlShenaifi et al, 2015;Shaalan et al, 2010;Hassan et al, 2014) or a dictionary (Attia et al, 2014;Zerrouki et al, 2014;Alkanhal et al, 2012). For the first technique, detecting whether a word is misspelled or not depends on morphological analyzers.…”

Section: Auto-correction Of Hamozamentioning

confidence: 99%

An enhanced automatic speech recognition system for Arabic

Menacer¹,

Mella²,

Fohr³

et al. 2017

Proceedings of the Third Arabic Natural Language Processing Workshop

View full text Add to dashboard Cite

Automatic speech recognition for Arabic is a very challenging task. Despite all the classical techniques for Automatic Speech Recognition (ASR), which can be efficiently applied to Arabic speech recognition, it is essential to take into consideration the language specificities to improve the system performance. In this article, we focus on Modern Standard Arabic (MSA) speech recognition. We introduce the challenges related to Arabic language, namely the complex morphology nature of the language and the absence of the short vowels in written text, which leads to several potential vowelization for each graphemes, which is often conflicting. We develop an ASR system for MSA by using Kaldi toolkit. Several acoustic and language models are trained. We obtain a Word Error Rate (WER) of 14.42 for the baseline system and 12.2 relative improvement by rescoring the lattice and by rewriting the output with the right hamoza above or below Alif.

show abstract

Arabic Spelling Correction using Supervised Learning

Cited by 15 publications

References 4 publications

Proceedings of the Second Workshop on Arabic Natural Language Processing

Proceedings of the Second Workshop on Arabic Natural Language Processing

Proceedings of the Third Arabic Natural Language Processing Workshop

An enhanced automatic speech recognition system for Arabic

Contact Info

Product

Resources

About