Automated arabic text classification with P‐Stemmer, machine learning, and a tailored news article taxonomy

Abstract

Arabic news articles in electronic collections are difficult to work with. Browsing by category is rarely supported. While helpful machine learning methods have been applied successfully to similar situations for English news articles, limited research has been completed to yield suitable solutions for Arabic news. In connection with a QNRF funded project to build digital library community and infrastructure in Qatar, we developed software for browsing a collection of about 237K Arabic news articles, which should be applicable to other Arabic news collections as well. We designed a simple taxonomy for Arabic news stories that is suitable for the needs in Qatar and other nations, is compatible with the subject codes of the International Press Telecommunications Council, and was enhanced with the aid of a librarian expert as well as five Arabic-speaking volunteers. We developed tailored stemming (i.e., a new Arabic light stemmer) and automatic classification methods (the best being binary SVM classifiers) to work with the taxonomy. Using evaluation techniques commonly used in the information retrieval community, including 10-fold cross-validation and the Wilcoxon signed-rank test, we showed that our approach to stemming and classification is superior to state-of-the-art techniques.

The Effect of Preprocessing on Arabic Document Categorization

“…He claimed that experiments proved that the stemming technique is not always effective for Arabic document categorization. His experiments [4] ' ' ' ' English SVM Toman et al [5] ' ' ' -English and Czech NB Chirawichitchai et al [9] -' ' ' Thai NB, DT, SVM Mesleh [10,11] ' ' -' Arabic SVM Duwairi et al [15] -' ' -Arabic KNN Kanan [14] -' ' -Arabic SVM, NB, RF Zaki et al [18] ' ' ' -Arabic KNN Al-Shargabi et al [12] -' --Arabic NB, SVM, J48 Khorsheed et al [16] -' -' Arabic KNN, NB, SVM, etc. Ababneh et al [17] ' ' --Arabic KNN…”

“…The Arabic language is a native language of the Arab states and the secondary language in a number of other countries [19]. More than 422 million people are able to speak Arabic, which makes this language the fifth most spoken language in the world, according to [14]. The alphabet of the Arabic language consists of 28 letters:…”

“…Uysal et al [7] N/F -+ (often) Pomikálek et al [6] N/F + + Méndez et al [8] N/F --Chirawichitchai et al [9] N/F N/F N/F Song et al [4] N/F + + Toman et al [5] N/F + -Mesleh [10,11] N/F N/F -Duwairi et al [15] N/F N/F + Kanan [14] 2015 N/F N/F + Zaki et al [18] N/F N/F + Al-Shargabi et al [12] N/F + N/F Khorsheed et al [16] N/F N/F N/F Ababneh et al [17] N/F N/F -…”

Image Classification in Arabic: Exploring Direct English to Arabic Translations

“…Several scholars have discussed the difficulties associated with developing natural language processing methods and algorithms for Arabic. These challenges include the ambiguity and complexity of Arabic (Kanan & Fox, 2016;Salloum, Al-emran, & Shaalan, 2016), the prevalence of several commonly used dialects in Arabic (Samih et al, 2017;Zalmout, Erdmann, & Habash, 2018), and the limited number of freely available datasets that can be used in the research and development for Arabic computational solutions (Zeroual & Lakhouaja, 2018). This study further investigates the complexity of Arabic and the problems associated with computational solutions that do not incorporate Arabic dialects.…”

“…It is relevant to note that in the domain of natural language processing, researchers often focus on developing solutions specific for a target language such as Arabic or Chinese for common tasks. These tasks include for example document classification and document summarization (Alanzi & Abuzeina, 2017;Al-Thubaity, Alhoshan, & Hazzaa, 2015;Kanan & Fox, 2016;Zhang, Xu, Su, & Xu, 2015). Therefore, focusing on novel methods to generate labels for images in Arabic is similarly important.…”