Automatic Arabic Sign Language Recognition: A Review, Taxonomy, Open Challenges, Research Roadmap and Future Directions

Abstract

Sign language is still the best communication mean between the deaf and hearing impaired citizens. Due to the advancements in technology, we are able to find various research attempts and efforts on Automatic Sign Language Recognition (ASLR) technology for many languages including the Arabic language. Such attempts have simplified and assisted the interpretation between spoken and sign languages. In fact, the technologies that translate between spoken and sign languages have become popular today. Being the first comprehensive and up-to-date review that studies the state-of-the-art ASLR in perspective to Arabic Sign Language Recognition (ArSLR), this review is a contribution to ArSLR research community. In this paper, the research background and fundamentals of ArSLR are provided. ArSLR research taxonomies, databases, open challenges, future research trends, and directions, and a roadmap to ArSLR research are presented. This review investigates two major taxonomies. The primary taxonomy that is related to the capturing mechanism of the gestures for ArSLR, which can be either a Vision-Based Recognition (VBR) approach or Sensor-Based Recognition (SBR) approach. The secondary taxonomy that is related to the type and task of the gestures for ArSLR, which can be either the Arabic alphabet, isolated words, or continuous sign language recognition. In addition, less research attempts have been directed towards Arabic continuous sign language recognition task compared to other tasks, which marks a research gap that can be considered by the research community. To the best of our knowledge, all previous research attempts and reviews on sign language recognition for ArSL used forehand signs. This shows that the backhand signs have not been considered for ArSL tasks, which creates another important research gap to be filled up. Therefore, we recommend more research initiatives to contribute to these gaps by using an SBR approach for signers' dependent and independent approaches.

Investigating The Use Of Leap Motion Controller In Recognition Of Arabic Sign Language

“…Opposite than other studies, the results indicated that the device has some issues. While previous studies [2], [12] claimed that the device has a great potential, our results reveal that the device has its limitations. When comparing between the studies, we found that the previous studies have focused on specific letters or context that are performed only by hands.…”

“…Recently, much research has been conducted on the use of leap motion controllers to detect and deal with ArSL [12]. Despite the momentum of using leap motion controllers to deal with ArSL, we have concerns about the correctness or completeness of the generated results.…”

Arabic Sign Language Gesture Classification Using Deer Hunting Optimization with Machine Learning Model

“…If a deaf individual attempts to express anything, they employ gestures for communication. Every symbol indicates a special letter, emotion, or word [7]. A stage was formed by signal combination, and a string of words invokes letters in spoken languages.…”

A Sign Language Recognition Method Relying On Convolutional Recurrent Neural Network

“…At present, many experts and scholars have conducted continuous and in-depth research on SLR methods and convolutional neural networks, and have achieved fruitful results. For example, researchers such as Al-Shamayleh A S proposed a very compact in-place gated CRNN for end-to-end multi-channel speech enhancement, which utilizes in-place convolution for frequency pattern extraction and reconstruction, in-place The features effectively preserve spatial cues in each frequency bin, and utilize a novel spectral restoration method to effectively improve speech quality by predicting magnitude masks, mappings, and phases [6]. Researchers such as Saleh BM proposed a robust SLR method based on deep learning, which represented multimodal information through texture maps to describe hand position and motion, and used this information as the basis for two three-stream and two-stream CNN models.…”