Morphological Segmentation and Part-of-Speech Tagging for the Arabic Heritage

Mohamed, Emad

doi:10.1145/3178459

Cited by 7 publications

(10 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our segmentation convention matches with (Aliwy, 2012;Mohamed, 2018;Habash et al, 2012) where clitics are split from words and the of notion of clitics is aligned as the syntactic units that can be assigned a POS tag and can occupy a node on the syntactic tree. It is also similar to the Penn Arabic Treebank (ATB) (Maamouri et al, 2004) with the exception of the definite article where we consider it as a clitic while in the ATB it is taken as a definiteness marker.…”

Section: Comparison Of Segmentation Conventionsmentioning

confidence: 97%

“…Abdelali et al (2016) developed a segmenter for their tool, Farasa, using SVM and trained on the ATB data with reported accuracy of 98.94%. Moreover, Mohamed (2018) developed a memorybased learning segmenter for Arabic religious texts trained on a manually annotated in-domain corpus of 27k words combined with the ATB data with reported accuracy of 95.70%.…”

Section: Related Workmentioning

confidence: 99%

“…not considered as a separate token), and therefore, most NLP applications based on this data model reflect this convention. In most other research efforts, such as (Abdelali et al, 2016;Aliwy, 2012;Mohamed, 2018;Habash et al, 2012), it is considered as a clitic and is segmented away from the nouns and adjectives they attach to.…”

Section: The Definite Article Dilemmamentioning

confidence: 99%

See 2 more Smart Citations

Segmentation for Domain Adaptation in Arabic

Attia

Elkahky

2019

Proceedings of the Fourth Arabic Natural Language Processing Workshop

View full text Add to dashboard Cite

Segmentation serves as an integral part in many NLP applications including Machine Translation, Parsing, and Information Retrieval. When a model trained on the standard language is applied to dialects, the accuracy drops dramatically. However, there are more lexical items shared by the standard language and dialects than can be found by mere surface word matching. This shared lexicon is obscured by a lot of cliticization, gemination, and character repetition. In this paper, we prove that segmentation and base normalization of dialects can help in domain adaptation by reducing data sparseness. Segmentation will improve a system performance by reducing the number of OOVs, help isolate the differences and allow better utilization of the commonalities. We show that adding a small amount of dialectal segmentation training data reduced OOVs by 5% and remarkably improves POS tagging for dialects by 7.37% f-score, even though no dialect-specific POS training data is included.

show abstract

Section: Comparison Of Segmentation Conventionsmentioning

confidence: 97%

Section: Related Workmentioning

confidence: 99%

Section: The Definite Article Dilemmamentioning

confidence: 99%

See 1 more Smart Citation

Segmentation for Domain Adaptation in Arabic

Attia

Elkahky

2019

Proceedings of the Fourth Arabic Natural Language Processing Workshop

View full text Add to dashboard Cite

show abstract

“…The recent studies of part of speech tagging include different aspects. For instance, [34] developed a part of speech tagger for the Arabic heritage. They scored an accuracy of 96.22%.…”

Section: Literature Reviewmentioning

confidence: 99%

Exploring the Performance of Tagging for the Classical and the Modern Standard Arabic

AbuZeina

Abdalbaset

2019

Advances in Fuzzy Systems

View full text Add to dashboard Cite

The part of speech (PoS) tagging is a core component in many natural language processing (NLP) applications. In fact, the PoS taggers contribute as a preprocessing step in various NLP tasks, such as syntactic parsing, information extraction, machine translation, and speech synthesis. In this paper, we examine the performance of a modern standard Arabic (MSA) based tagger for the classical (i.e., traditional or historical) Arabic. In this work, we employed the Stanford Arabic model tagger to evaluate the imperative verbs in the Holy Quran. In fact, the Stanford tagger contains 29 tags; however, this work experimentally evaluates just one that is the VB ≡ imperative verb. The testing set contains 741 imperative verbs, which appear in 1,848 positions in the Holy Quran. Despite the previously reported accuracy of the Arabic model of the Stanford tagger, which is 96.26% for all tags and 80.14% for unknown words, the experimental results show that this accuracy is only 7.28% for the imperative verbs. This result promotes the need for further research to expose why the tagging is severely inaccurate for classical Arabic. The performance decline might be an indication of the necessity to distinguish between training data for both classical and MSA Arabic for NLP tasks.

show abstract

“…However, when we use these tools to segment pre-MSA texts, the quality drops significantly. MADAMIRA [10], the best known Arabic NLP system, which has an accuracy of over 98% on MSA, has an accuracy of 94.7% on a Classical Arabic test set [7]. The main difference between Classical Arabic and MSA lies in their different vocabularies.…”

Section: Introductionmentioning

confidence: 99%

Arabic-SOS

Mohamed

Sayyed

2019

Proceedings of the 3rd International Conference on Digital Access to Textual Cultural Heritage

Self Cite

View full text Add to dashboard Cite

While morphological segmentation has always been a hot topic in Arabic, due to the morphological complexity of the language and the orthography, most effort has focused on Modern Standard Arabic. In this paper, we focus on pre-MSA texts. We use the Gradient Boosting algorithm to train a morphological segmenter with a corpus derived from Al-Manar, a late 19th/early 20th century magazine that focused on the Arabic and Islamic heritage. Since most of the cultural heritage Arabic available suffers from substandard orthography, we have trained a machine learner to standardize the text. Our segmentation accuracy reaches 98.47%, and the orthography standardization an F-macro of 0.98 and an F-micro of 0.99. We also produce stemming as a by-product of segmentation.

show abstract

Morphological Segmentation and Part-of-Speech Tagging for the Arabic Heritage

Cited by 7 publications

References 7 publications

Segmentation for Domain Adaptation in Arabic

Segmentation for Domain Adaptation in Arabic

Exploring the Performance of Tagging for the Classical and the Modern Standard Arabic

Arabic-SOS

Contact Info

Product

Resources

About