A Modern Approach for Sign Language Interpretation Using Convolutional Neural Network

Paul, Pias; Bhuiya, Moh. Anwar-Ul-Azim; Ullah, Md. Ayat; Saqib, Molla Nazmus; Mohammed, Nabeel; Momen, Sifat

doi:10.1007/978-3-030-29894-4_35

Cited by 14 publications

(3 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The model achieved an accuracy of 94.8%. Furthermore, Paul et al [48] proposed two CNNs to categorize 24 static signs from ASL. The work is based on the ASL Finger Spelling dataset and attained an accuracy of 86.52% and 85.88% on RGB images.…”

Section: Related Workmentioning

confidence: 99%

Helping Hearing-Impaired in Emergency Situations: A Deep Learning-Based Approach

et al. 2022

View full text Add to dashboard Cite

Hearing-impaired people use sign language to express their thoughts and emotions and reinforce information delivered in daily conversations. Though they make a significant percentage of any population, the majority of people can't interact with them due to limited or no knowledge of sign languages. Sign language recognition aims to detect the significant motions of the human body, especially hands, analyze them and understand them. Such systems may become life-saving when hearing-challenged people are in desperate situations like heart attacks, accidents, etc. In the present study, deep learning-based hand gesture recognition models are developed to accurately predict the emergency signs of Indian Sign Language (ISL). The dataset used contains the videos for eight different emergency situations. Several frames were extracted from the videos and are fed to three different models. Two models are designed for classification, while one is an object detection model, applied after annotating the frames. The first model consists of a three-dimensional convolutional neural network (3D CNN), while the second comprises of a pre-trained VGG-16 and a recurrent neural network with a long short-term memory (RNN-LSTM) scheme. The last model is based on YOLO (You Only Look Once) v5, an advanced object detection algorithm. The prediction accuracies of the classification models were 82% and 98%, respectively. YOLO based model outperformed the rest and achieved an impressive mean average precision of 99.6%.

show abstract

Section: Related Workmentioning

confidence: 99%

Helping Hearing-Impaired in Emergency Situations: A Deep Learning-Based Approach

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The initial dataset used in this work consists of 8057 bird images which were further transformed as the training data [42] through rotation, horizontal and vertical shift, shear, zoom, and horizontal flip to increase the number of training images by following [34,43,44]. Moreover, the technique used for re-scaling in [45] was applied to the pixel range from [0, . .…”

Section: Data Augmentation and Class Weightsmentioning

confidence: 99%

An Improved Transfer-Learning for Image-Based Species Classification of Protected Indonesians Birds

Yang¹,

Harjoseputro²,

Hu³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

This research proposed an improved transfer-learning bird classification framework to achieve a more precise classification of Protected Indonesia Birds (PIB) which have been identified as the endangered bird species. The framework takes advantage of using the proposed sequence of Batch Normalization Dropout Fully-Connected (BNDFC) layers to enhance the baseline model of transfer learning. The main contribution of this work is the proposed sequence of BNDFC that can be applied to any Convolutional Neural Network (CNN) based model to improve the classification accuracy, especially for image-based species classification problems. The experiment results show that the proposed sequence of BNDFC layers outperform other combination of BNDFC. The addition of BNDFC can improve the model's performance across ten different CNN-based models. On average, BNDFC can improve by approximately 19.88% in Accuracy, 24.43% in F-measure, 17.93% in G-mean, 23.41% in Sensitivity, and 18.76% in Precision. Moreover, applying fine-tuning (FT) is able to enhance the accuracy by 0.85% with a smaller validation loss of 18.33% improvement. In addition, MobileNetV2 was observed to be the best baseline model with the lightest size of 35.9 MB and the highest accuracy of 88.07% in the validation set.

show abstract

“…As hand signs continue to play an increasingly prominent role in our daily lives, researchers have developed various techniques to enhance their identification. Among these, Convolutional Neural Networks (CNNs) have emerged as one of the most popular and effective approaches for extracting information from images of hands [4][5][6][7][8].…”

Section: Introductionmentioning

confidence: 99%

Ensemble Transfer Learning for Hand-sign Digit Image Classification

Andi Muhammad Amil Siddik,

Ainun Mawaddah Abdal,

Armin Lawi

et al. 2024

ARASET

View full text Add to dashboard Cite

Hand sign classification is a challenging task in image processing and machine learning. Robust learning algorithms are essential to achieve optimal performance. Ensemble transfer learning, a technique that combines ensemble learning and transfer learning, is a promising approach to improve classification model performance. This study investigates the use of ensemble transfer learning for hand sign classification. The Sign Language Digits Dataset, which contains ten distinct handwritten image types, was used to evaluate the performance of three architectures: ResNet-50, VGG-19, and Ensemble Transfer Learning (a fusion of ResNet-50 and VGG-19). The results showed that all three architectures performed well, but Ensemble Transfer Learning achieved the best performance with an accuracy of 96.12%, precision of 96.06%, recall of 96.21%, and F1 score of 96.07%. This suggests that ensemble transfer learning can effectively enhance model performance in image hand sign classification. The study also found that combining ResNet-50 and VGG-19 in Ensemble Transfer Learning yielded superior results compared to individual models. This is because ensemble transfer learning can leverage the strengths of both models to improve the overall performance. The findings of this study highlight the significance of employing ensemble transfer learning techniques to enhance accuracy and reliability in hand sign image classification.

show abstract

A Modern Approach for Sign Language Interpretation Using Convolutional Neural Network

Cited by 14 publications

References 20 publications

Helping Hearing-Impaired in Emergency Situations: A Deep Learning-Based Approach

Helping Hearing-Impaired in Emergency Situations: A Deep Learning-Based Approach

An Improved Transfer-Learning for Image-Based Species Classification of Protected Indonesians Birds

Ensemble Transfer Learning for Hand-sign Digit Image Classification

Contact Info

Product

Resources

About