Vision-based Pakistani sign language recognition using bag-of-words and support vector machines

Mirza, Muhammad Shaheer; Munaf, Sheikh Muhammad; Azim, Fahad; Ali, Shahid; Khan, Saad Jawaid

doi:10.1038/s41598-022-15864-6

Cited by 7 publications

(3 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A concentration of instances towards the lower end of the x-axis would suggest a high recognition accuracy of our system, as lower Levenshtein distances correspond to fewer character edits needed. Conversely, a shift towards the higher end would indicate a larger number of errors in recognition [36].…”

Section: Resultsmentioning

confidence: 99%

Two Dimensional Deep CNN Model for Vision-based Fingerspelling Recognition System

Kozhamkulova,

Nurlybaeva,

Kuntunova

et al. 2023

IJACSA

View full text Add to dashboard Cite

This paper presents a novel approach to fingerspelling recognition in real-time, utilizing a twodimensional Convolutional Neural Network (2D CNN). Existing recognition systems often fall short in real-world conditions due to variations in illumination, background, and user-specific characteristics. Our method addresses these challenges, delivering significantly improved performance. Leveraging a robust 2D CNN architecture, the system processes image sequences representing the dynamic nature of fingerspelling. We focus on low-level spatial features and temporal patterns, thereby ensuring a more accurate capture of the intricate nuances of fingerspelling. Additionally, the incorporation of real-time video feed enhances the system's responsiveness. We validate our model through comprehensive experiments, showcasing its superior recognition rate over current methods. In scenarios involving varied lighting, different backgrounds, and distinct user behaviors, our system consistently outperforms. The findings demonstrate that the 2D CNN approach holds promise in improving fingerspelling recognition, thereby aiding communication for the hearing-impaired community. This work paves the way for further exploration of deep learning applications in real-time sign language interpretation. This research bears profound implications for accessibility and inclusivity in communication technology.

show abstract

Section: Resultsmentioning

confidence: 99%

Two Dimensional Deep CNN Model for Vision-based Fingerspelling Recognition System

Kozhamkulova,

Nurlybaeva,

Kuntunova

et al. 2023

IJACSA

View full text Add to dashboard Cite

show abstract

“…Many types of SLs have been used in previous studies based on countries and spoken languages. Languages used in SL research are numerous including American [16,22,[41][42][43][44][45][46], Mexican [47], Arabic [3,7,10,11,39,40,48,49], Algerian [50], Pakistani [51], Indian [52][53][54], Bangla [55], Chinese [56,57], Indonesian [58], Italian [23], Peruvian [59], Turkish [34], and Urdu [24]. Based on [60], which analyzed SL classification research studies published from 2014 to 2021, another breakdown of literature works was presented according to the local variation in the SL they referred to.…”

Section: Overview Of Sl Classification Research Based On Languagesmentioning

confidence: 99%

“…Samples in SL datasets can be grayscale or RGB format. Any researcher interested in this field can obtain an SL dataset from online repositories such as Kaggle or by building a dataset [3,[33][34][35]51,63,64]. Table 3 summarizes various datasets in terms of isolated vs. continuous, static vs. dynamic, language, recognition model, lexicon size, acquisition mode, signs, signers, single vs. double handed, and background conditions.…”

Section: Overview Of Sl Classification Research Based On Datasetsmentioning

confidence: 99%

Vision Transformers and Transfer Learning Approaches for Arabic Sign Language Recognition

Alharthi,

Alzahrani

2023

Applied Sciences

View full text Add to dashboard Cite

Sign languages are complex, but there are ongoing research efforts in engineering and data science to recognize, understand, and utilize them in real-time applications. Arabic sign language recognition (ArSL) has been examined and applied using various traditional and intelligent methods. However, there have been limited attempts to enhance this process by utilizing pretrained models and large-sized vision transformers designed for image classification tasks. This study aimed to create robust transfer learning models trained on a dataset of 54,049 images depicting 32 alphabets from an ArSL dataset. The goal was to accurately classify these images into their corresponding Arabic alphabets. This study included two methodological parts. The first one was the transfer learning approach, wherein we utilized various pretrained models namely MobileNet, Xception, Inception, InceptionResNet, DenseNet, and BiT, and two vision transformers namely ViT, and Swin. We evaluated different variants from base-sized to large-sized pretrained models and vision transformers with weights initialized from the ImageNet dataset or otherwise randomly. The second part was the deep learning approach using convolutional neural networks (CNNs), wherein several CNN architectures were trained from scratch to be compared with the transfer learning approach. The proposed methods were evaluated using the accuracy, AUC, precision, recall, F1 and loss metrics. The transfer learning approach consistently performed well on the ArSL dataset and outperformed other CNN models. ResNet and InceptionResNet obtained a comparably high performance of 98%. By combining the concepts of transformer-based architecture and pretraining, ViT and Swin leveraged the strengths of both architectures and reduced the number of parameters required for training, making them more efficient and stable than other models and existing studies for ArSL classification. This demonstrates the effectiveness and robustness of using transfer learning with vision transformers for sign language recognition for other low-resourced languages.

show abstract

Pakistan sign language recognition: leveraging deep learning models with limited dataset

Hamza

Wali

2023

Machine Vision and Applications

View full text Add to dashboard Cite

Vision-based Pakistani sign language recognition using bag-of-words and support vector machines

Cited by 7 publications

References 34 publications

Two Dimensional Deep CNN Model for Vision-based Fingerspelling Recognition System

Two Dimensional Deep CNN Model for Vision-based Fingerspelling Recognition System

Vision Transformers and Transfer Learning Approaches for Arabic Sign Language Recognition

Pakistan sign language recognition: leveraging deep learning models with limited dataset

Contact Info

Product

Resources

About