ArSign: Toward a Mobile Based Arabic Sign Language Translator Using LMC

Kammoun, Slim; Darwish, Dawlat; Althubeany, Hanan; Alfull, Reem

doi:10.1007/978-3-030-49108-6_7

Cited by 5 publications

(2 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The positions were classified using the Adaboost method. The handcrafted features systems were not robust ( Alnahhas et al, 2020 ; Deriche, Aliyu & Mohandes, 2019 ; Kammoun et al, 2020 ; Bird, 2022 ). Alternatively, Bansal, Wadhawan & Goel (2022) optimize the histogram of gradient features using embedded particle swamp optimization.…”

Section: Literature Reviewmentioning

confidence: 99%

Intelligent real-life key-pixel image detection system for early Arabic sign language learners

Alamri,

Rehman,

Abdullahi

et al. 2024

PeerJ Computer Science

View full text Add to dashboard Cite

Lack of an effective early sign language learning framework for a hard-of-hearing population can have traumatic consequences, causing social isolation and unfair treatment in workplaces. Alphabet and digit detection methods have been the basic framework for early sign language learning but are restricted by performance and accuracy, making it difficult to detect signs in real life. This article proposes an improved sign language detection method for early sign language learners based on the You Only Look Once version 8.0 (YOLOv8) algorithm, referred to as the intelligent sign language detection system (iSDS), which exploits the power of deep learning to detect sign language-distinct features. The iSDS method could overcome the false positive rates and improve the accuracy as well as the speed of sign language detection. The proposed iSDS framework for early sign language learners consists of three basic steps: (i) image pixel processing to extract features that are underrepresented in the frame, (ii) inter-dependence pixel-based feature extraction using YOLOv8, (iii) web-based signer independence validation. The proposed iSDS enables faster response times and reduces misinterpretation and inference delay time. The iSDS achieved state-of-the-art performance of over 97% for precision, recall, and F1-score with the best mAP of 87%. The proposed iSDS method has several potential applications, including continuous sign language detection systems and intelligent web-based sign recognition systems.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Intelligent real-life key-pixel image detection system for early Arabic sign language learners

Alamri,

Rehman,

Abdullahi

et al. 2024

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…This section explores an influx of related publications on sign language recognition techniques. According to [20][21][22][23], the implementation of a sign language recognition system can be carried out either by using a sensor-based approach, an image-based approach, or both approaches (hybrid), as can be seen in Figure 4. In the sensor-based system technique, the user wears a specialized glove equipped with multiple sensors and wires.…”

Section: Related Workmentioning

confidence: 99%

Deep Learning Technology to Recognize American Sign Language Alphabet

Alsharif,

Altaher,

Altaher

et al. 2023

Sensors

View full text Add to dashboard Cite

Historically, individuals with hearing impairments have faced neglect, lacking the necessary tools to facilitate effective communication. However, advancements in modern technology have paved the way for the development of various tools and software aimed at improving the quality of life for hearing-disabled individuals. This research paper presents a comprehensive study employing five distinct deep learning models to recognize hand gestures for the American Sign Language (ASL) alphabet. The primary objective of this study was to leverage contemporary technology to bridge the communication gap between hearing-impaired individuals and individuals with no hearing impairment. The models utilized in this research include AlexNet, ConvNeXt, EfficientNet, ResNet-50, and VisionTransformer were trained and tested using an extensive dataset comprising over 87,000 images of the ASL alphabet hand gestures. Numerous experiments were conducted, involving modifications to the architectural design parameters of the models to obtain maximum recognition accuracy. The experimental results of our study revealed that ResNet-50 achieved an exceptional accuracy rate of 99.98%, the highest among all models. EfficientNet attained an accuracy rate of 99.95%, ConvNeXt achieved 99.51% accuracy, AlexNet attained 99.50% accuracy, while VisionTransformer yielded the lowest accuracy of 88.59%.

show abstract

Enhanced Weak Spatial Modeling Through CNN-Based Deep Sign Language Skeletal Feature Transformation

Alamri,

Bala Abdullahi,

Khan

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Recent sign language skeletal-based feature models (SLSm) consist of various distracting coordinates that lead to complex deep-learning modeling. However, SLSm is not purely a spatial-temporal coordinate arrangement problem; it is also limited by human dynamics and feature aggregations. The objectives of this work are twofold: (a) to transform the skeletal features of the SLSm model to address the problem of variations in viewpoint and changes across features of repeated signs due to human dynamics, and (b) to exploit the potential of exhaustive searching in dropping distracting features to prevent complex deep learning modeling. Method: We propose a transformed skeletal feature-based model (SCT) from a feature thresholding theory. We first extract the hand-skeletal joint-related features relevant to the coordinates and positions of the hand transcription that efficiently capture human dynamics. The extracted features are transformed into a subset of a predefined threshold and fed into the proposed ensemble exhaustive feature searching. The searched features are transformed into their equivalent deep input image sequences. Outcomes: By leveraging the skeletal-based transformed and deep spatial features, the proposed method demonstrates robust performance in sign language recognition, surpassing recent deep learning models in accuracy and simplicity. The proposed skeletal features demonstrate superiority in learning complex hand gestures of public data sets, improving accuracy by more than 2%.INDEX TERMS Human-computer interaction, End-to-end deep neural network, Multimodal data interaction, Hand gestures, Sign language recognition, and Pattern recognition.

show abstract

ArSign: Toward a Mobile Based Arabic Sign Language Translator Using LMC

Cited by 5 publications

References 27 publications

Intelligent real-life key-pixel image detection system for early Arabic sign language learners

Intelligent real-life key-pixel image detection system for early Arabic sign language learners

Deep Learning Technology to Recognize American Sign Language Alphabet

Enhanced Weak Spatial Modeling Through CNN-Based Deep Sign Language Skeletal Feature Transformation

Contact Info

Product

Resources

About