Towards enhancing retrieval effectiveness of search engines for diacritisized Arabic documents

Abstract

The majority of Arabic text available on the web is written without short vowels (diacritics). Diacritics are commonly used in religious scripts such as the holy Quran (the book of Islam), Al-Hadith (the teachings of Prophet Mohammad (PBUH)), children's literature, and in some words where ambiguity of articulation might arise. Internet Arabic users might lose credible sources of Arabic text to be retrieved if they could not match the correct diacritical marks attached to the words in the collection. However, typing the diacritical marks is very annoying and time consuming. The other way around, is to ignore these marks and fall into the problem of ambiguity. Previous work suggested pre-processing of Arabic text to remove these diacritical marks before indexing. Consequently, there are noticeable discrepancies when searching the web for Arabic text using international search engines such as Google and yahoo. In this article, we propose a framework to enhance the retrieval effectiveness of search engines to search for diacritic and diacritic-less Arabic text through query expansion techniques. We used a rule-based stemmer and a semantic relational database compiled in an experimental thesaurus to do the expansion. We tested our approach on the scripts of the Quran. We found that query expansion for searching Arabic text is promising and it is likely that the efficiency can be further improved by advanced natural language processing tools.

A Comparative Survey on Arabic Stemming: Approaches and Challenges

“…In his study for the Holy Quran, Hammo [24] stated that most of the failing cases of Khoja when it was used to stem words of the Holy book, were occurred when stemming proper names such as the names of Prophets, angels, ancient cities, places and people, numerals, as well as words with the diacritical mark sha-dda.…”

A Comparative Study on Arabic Stemmers

“…Stemming is the process of correlating several terms onto one common representation in the base form [16]. It minimizes the index size because it has the advantage of reducing storage requirements by eliminating the redundant words.…”

“…Stemming uses morphological heuristics in order to remove affixes from words and the processing cost is relatively low. For those reasons, the stemming is important and highly attractive for many natural language processing (NLP) fields such as: information retrieval (IR), question answering (QA), information extraction (IE), machine translation (MT), text summarizations (TS), Text Classification (TC), Text Clustering (TClu), Text segmentation (TS), Indexing (Ind), and Automatic Speech Recognition (ASR) [16]. There are many developed algorithmic stemming and various morphological analysis approaches to achieve morphologically related forms combined under the same stem using stemmer [14].…”

“…Arabic stemming is more complicated than stemming in any European language such as English [16]. The usage of affixes in English language is less complex compared to Arabic language because for English language, stemmers are only concerned with the removal of the suffixes [4].…”

Enhanced Search Scheme Precision and Performance using a GA Approach with Application to Arabic Content

“…The growing number of Arabic documents on the Web signals the need for advanced and improved Web search engines that retrieve related Arabic documents with high correctness and less time based on user requests. Precision, percentage of the retrieved related-documents and recall are measures used to determine the IR system's effectiveness and correctness (Abdelmgeid, 2007;Hammo, 2009).…”