Arabic Sign Language Recognition System for Alphabets Using Machine Learning Techniques

Abstract

In recent years, the role of pattern recognition in systems based on human computer interaction (HCI) has spread in terms of computer vision applications and machine learning, and one of the most important of these applications is to recognize the hand gestures used in dealing with deaf people, in particular to recognize the dashed letters in surahs of the Quran. In this paper, we suggest an Arabic Alphabet Sign Language Recognition System (AArSLRS) using the vision-based approach. The proposed system consists of four stages: the stage of data processing, preprocessing of data, feature extraction, and classification. The system deals with three types of datasets: data dealing with bare hands and a dark background, data dealing with bare hands, but with a light background, and data dealing with hands wearing dark colored gloves. AArSLRS begins with obtaining an image of the alphabet gestures, then revealing the hand from the image and isolating it from the background using one of the proposed methods, after which the hand features are extracted according to the selection method used to extract them. Regarding the classification process in this system, we have used supervised learning techniques for the classification of 28-letter Arabic alphabet using 9240 images. We focused on the classification for 14 alphabetic letters that represent the first Quran surahs in the Quranic sign language (QSL). AArSLRS achieved an accuracy of 99.5% for the K-Nearest Neighbor (KNN) classifier.

Human-Computer Interaction with Hand Gesture Recognition Using ResNet and MobileNet

“…In [ 29 ], as an example, a team of academics from the Al-Azhar University in Cairo, Egypt, and the King Khalid University in Abha, Saudi Arabia, presented an Arabic sign language recognition system for alphabets using machine learning methods. They experimented on a collection of 2,800 photographs and 28 classes, with 10 persons considering the 28 distinct alphabets.…”

An Educational Arabic Sign Language Mobile Application for Children with Hearing Impairment

“…In spite of that, grammar is not followed as in the spoken and the written Arabic language, but facial expressions or other ways can express the particles and the verb tenses. The expressions, words, and alphabet can be expressed using finger spelling and cued speech as a communication method among people with hearing impairment [6], [18].…”

“…Many mobile applications have been developed to teach the various sign languages with most dedicated to teach the American and English versions [6], [20]. As reported [10], few mobile applications have been developed to teach ArSL.…”

“…For instance, there is Italian sign language (ISL), Chinese sign language (CSL), French sign language (FSL), and Arabic sign language (ArSL), to list a few. Within ArSL, various sign languages have been developed by different countries due to differences in the Arabic dialects [6], [7]. For example, there is Emirati sign language, Jordanian sign language, Tunisian sign language, and Egyptian sign language, to list a few.…”

“…Various methodologies and applications have been proposed to develop and embed different technologies within the sign language learning process. The aim of these systems is to improve the interaction and learning between the deaf and other people through the daily practice of communication [5], [6], [10], [23], [30], [34]. Many projects have used PC-based or mobile-based applications to enhance communication between people with disabilities [27], [33] and in particular between deaf people and nondeaf people to improve the experience of learning sign languages [30].…”