Isolated Video-Based Arabic Sign Language Recognition Using Convolutional and Recursive Neural Networks

Boukdir, Abdelbasset; Benaddy, Mohamed; Ellahyani, Ayoub; Meslouhi, Othmane El; Kardouchi, Mustapha

doi:10.1007/s13369-021-06167-5

Cited by 19 publications

(11 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed MobileNetV2-LSTM-SelfMLP (q = 3) model on the segmented dataset outperforms other contemporary literature in the Arabic Sign Language domain and a comparative analysis is given in Table 8 . In [ 32 ], the authors used a 2D convolutional recurrent neural network (2DCRNN) and a 3D convolutional neural network (3DCNN) for Arabic Sign Language recognition. The best accuracy of 99% was achieved using a 3DCNN, but the model was heavier and had a slower inference time than our proposed model.…”

Section: Resultsmentioning

confidence: 99%

“…The two-dimensional (2D) convolutional recurrent neural network (2D-CRNN) and the 3D convolutional neural network (3D-CNN) were also adopted by researchers to achieve 92% and 99% accuracy, but the dataset consisted of 224 videos of five signers executing 56 distinct signs [ 32 ]. Three-dimensional (3D) GS-NET is another research in Arabic Sign Language recognition, where the system can recognize signs from RGB videos [ 33 ].…”

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

Signer-Independent Arabic Sign Language Recognition System Using Deep Learning Model

Podder,

Ezeddin,

Chowdhury

et al. 2023

Sensors

View full text Add to dashboard Cite

Every one of us has a unique manner of communicating to explore the world, and such communication helps to interpret life. Sign language is the popular language of communication for hearing and speech-disabled people. When a sign language user interacts with a non-sign language user, it becomes difficult for a signer to express themselves to another person. A sign language recognition system can help a signer to interpret the sign of a non-sign language user. This study presents a sign language recognition system that is capable of recognizing Arabic Sign Language from recorded RGB videos. To achieve this, two datasets were considered, such as (1) the raw dataset and (2) the face–hand region-based segmented dataset produced from the raw dataset. Moreover, operational layer-based multi-layer perceptron “SelfMLP” is proposed in this study to build CNN-LSTM-SelfMLP models for Arabic Sign Language recognition. MobileNetV2 and ResNet18-based CNN backbones and three SelfMLPs were used to construct six different models of CNN-LSTM-SelfMLP architecture for performance comparison of Arabic Sign Language recognition. This study examined the signer-independent mode to deal with real-time application circumstances. As a result, MobileNetV2-LSTM-SelfMLP on the segmented dataset achieved the best accuracy of 87.69% with 88.57% precision, 87.69% recall, 87.72% F1 score, and 99.75% specificity. Overall, face–hand region-based segmentation and SelfMLP-infused MobileNetV2-LSTM-SelfMLP surpassed the previous findings on Arabic Sign Language recognition by 10.970% accuracy.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

Signer-Independent Arabic Sign Language Recognition System Using Deep Learning Model

Podder,

Ezeddin,

Chowdhury

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…The incorporation of gesture recognition technology, namely in the domain of Arabic Sign Language (ArSL), has represented a notable advancement in enabling communication between those with speech impairments and computer systems [14]. This technological innovation is crucial for the identification and comprehension of ArSL, which possesses its distinct repertoire of gestures and facial expressions [15,16]. Utilizing these strategies significantly reduces Deaf ASL users' communication barriers and increases the ability for them to participate in a wide range of professional and social settings.…”

Section: Related Workmentioning

confidence: 99%

“…Researchers obtained a 92% accuracy rate by utilizing a twodimensional Convolutional Recurrent Neural Network (2D-CRNN) and a 99% accuracy rate with a three-dimensional Convolutional Neural Network (3D-CNN). These models were tested on a dataset consisting of 224 movies, where five signers performed 56 distinct signs [15].…”

Section: Related Workmentioning

confidence: 99%

Efficient YOLO Based Deep Learning Model for Arabic Sign Language Recognition

Ahmadi,

Mohammad,

Dawsari

2024

Preprint

View full text Add to dashboard Cite

Verbal communication is the dominant form of self-expression and interpersonal communication. Speech is a considerable obstacle for individuals with disabilities, including those who are deaf, hard of hearing, mute, or nonverbal. Consequently, these individuals depend on sign language to communicate with others. Sign Language is a complex system of gestures and visual cues that facilitate the inclusion of individuals into vocal communication groups. In this manuscript a novel technique proposed using deep learning to recognize the Arabic Sign language (ArSL) accurately. Through this advanced system, the objective is to help in communication between the hearing and deaf community. The proposed mechanism relies on advanced attention mechanisms, and state-of-art Convolutional Neural Network (CNN) architectures with the robust YOLO object detection model that highly improves the implementation and accuracy of ArSL recognition. In our proposed method, we integrate the self-attention block, channel attention module, spatial attention module, and cross-convolution module into the features processing, and the ArSL recognition accuracy reaches 98.9%. The recognition accuracy of our method is significantly improved with higher detection rate. The presented approach showed significant improvement as compared with the conventional techniques with a precision rate of 0.9. For the mAP@0.5, the mAP score is 0.9909 while for the mAP@0.5:0.95 and the results tops all the state-of-the-art techniques. This shows that the model has the great capability to accurately detect and classify complex multiple ArSL signs. The model provides a unique way of linking people and improving the communication strategy while also promoting the social inclusion of deaf people in the Arabic region.

show abstract

“…Communication is classified into verbal and nonverbal forms, and its core is to exchange data between the sender and the receiver [ 1 ]. As the component of communication, verbal and non-verbal forms are considered spontaneous and disguised spontaneous communications, the initial one is demonstrated as an intentional communication from the motivation emotional state, and the last one is demonstrated as an instinctive intentional strategic operation [ 2 ]. Communication is an indispensable tool in the existence of human beings.…”

Section: Introductionmentioning

confidence: 99%

Atom Search Optimization with Deep Learning Enabled Arabic Sign Language Recognition for Speaking and Hearing Disability Persons

et al. 2022

View full text Add to dashboard Cite

Sign language has played a crucial role in the lives of impaired people having hearing and speaking disabilities. They can send messages via hand gesture movement. Arabic Sign Language (ASL) recognition is a very difficult task because of its high complexity and the increasing intraclass similarity. Sign language may be utilized for the communication of sentences, letters, or words using diverse signs of the hands. Such communication helps to bridge the communication gap between people with hearing impairment and other people and also makes it easy for people with hearing impairment to express their opinions. Recently, a large number of studies have been ongoing in developing a system that is capable of classifying signs of dissimilar sign languages into the given class. Therefore, this study designs an atom search optimization with a deep convolutional autoencoder-enabled sign language recognition (ASODCAE-SLR) model for speaking and hearing disabled persons. The presented ASODCAE-SLR technique mainly aims to assist the communication of speaking and hearing disabled persons via the SLR process. To accomplish this, the ASODCAE-SLR technique initially pre-processes the input frames by a weighted average filtering approach. In addition, the ASODCAE-SLR technique employs a capsule network (CapsNet) feature extractor to produce a collection of feature vectors. For the recognition of sign language, the DCAE model is exploited in the study. At the final stage, the ASO algorithm is utilized as a hyperparameter optimizer which in turn increases the efficacy of the DCAE model. The experimental validation of the ASODCAE-SLR model is tested using the Arabic Sign Language dataset. The simulation analysis exhibit the enhanced performance of the ASODCAE-SLR model compared to existing models.

show abstract

Isolated Video-Based Arabic Sign Language Recognition Using Convolutional and Recursive Neural Networks

Cited by 19 publications

References 31 publications

Signer-Independent Arabic Sign Language Recognition System Using Deep Learning Model

Signer-Independent Arabic Sign Language Recognition System Using Deep Learning Model

Efficient YOLO Based Deep Learning Model for Arabic Sign Language Recognition

Atom Search Optimization with Deep Learning Enabled Arabic Sign Language Recognition for Speaking and Hearing Disability Persons

Contact Info

Product

Resources

About