SignExplainer: An Explainable AI-Enabled Framework for Sign Language Recognition With Ensemble Learning

Kothadiya, Deep; Bhatt, Chintan; Rehman, Amjad; Alamri, Faten S.; Saba, Tanzila

doi:10.1109/access.2023.3274851

Cited by 16 publications

(5 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 9 represents a comparative analysis of the proposed sequential attention model against other state-of-the-art convolution models such as VGG19, Densenet121, and InceptionV3, and different optimizers such as Gradient Descent, Stochastic Gradient Decent (SDG), and Adam [ 33 ]. Table 8 shows a comparative analysis with different optimizers and different deep learning models.…”

Section: Resultsmentioning

confidence: 99%

“…Some other tests have been run with various train test split ratios, as shown in Figure 8. Figure 9 represents a comparative analysis of the proposed sequential attention model against other state-of-the-art convolution models such as VGG19, Densenet121, and In-ceptionV3, and different optimizers such as Gradient Descent, Stochastic Gradient Decent (SDG), and Adam [33].…”

Section: Experiments and Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Enhancing Fingerprint Liveness Detection Accuracy Using Deep Learning: A Comprehensive Study and Novel Approach

et al. 2023

Self Cite

View full text Add to dashboard Cite

Liveness detection for fingerprint impressions plays a role in the meaningful prevention of any unauthorized activity or phishing attempt. The accessibility of unique individual identification has increased the popularity of biometrics. Deep learning with computer vision has proven remarkable results in image classification, detection, and many others. The proposed methodology relies on an attention model and ResNet convolutions. Spatial attention (SA) and channel attention (CA) models were used sequentially to enhance feature learning. A three-fold sequential attention model is used along with five convolution learning layers. The method’s performances have been tested across different pooling strategies, such as Max, Average, and Stochastic, over the LivDet-2021 dataset. Comparisons against different state-of-the-art variants of Convolutional Neural Networks, such as DenseNet121, VGG19, InceptionV3, and conventional ResNet50, have been carried out. In particular, tests have been aimed at assessing ResNet34 and ResNet50 models on feature extraction by further enhancing the sequential attention model. A Multilayer Perceptron (MLP) classifier used alongside a fully connected layer returns the ultimate prediction of the entire stack. Finally, the proposed method is also evaluated on feature extraction with and without attention models for ResNet and considering different pooling strategies.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Experiments and Resultsmentioning

confidence: 99%

Enhancing Fingerprint Liveness Detection Accuracy Using Deep Learning: A Comprehensive Study and Novel Approach

et al. 2023

Self Cite

View full text Add to dashboard Cite

show abstract

“…The vision transformer in [34] used eight layers of the encoder with four heads with seven million parameters, which is computationally expensive. The authors in [21] have used ensemble network architecture employing ResNet50 with an attention module with more training parameters and epochs.…”

Section: Time Complexity and Order Of The Proposed Methodsmentioning

confidence: 99%

“…Figure 2 shows a sample of the sign gesture images taken from this image database. The static ISL dataset [21] comprises 36 sign gestures of digits (0-9) and alphabet (A-Z), with more than 1200 sample images in each class captured using a web camera. There is a total of 34,554 images in the database having a size of 250 × 250 with augmentation to introduce variability in the data.…”

Section: Datasetmentioning

confidence: 99%

Fusion of Attention-Based Convolution Neural Network and HOG Features for Static Sign Language Recognition

Kumari,

Anand

2023

Applied Sciences

View full text Add to dashboard Cite

The deaf and hearing-impaired community expresses their emotions, communicates with society, and enhances the interaction between humans and computers using sign language gestures. This work presents a strategy for efficient feature extraction that uses a combination of two different methods that are the convolutional block attention module (CBAM)-based convolutional neural network (CNN) and standard handcrafted histogram of oriented gradients (HOG) feature descriptor. The proposed framework aims to enhance accuracy by extracting meaningful features and resolving issues like rotation, similar hand orientation, etc. The HOG feature extraction technique provides a compact feature representation that signifies meaningful information about sign gestures. The CBAM attention module is incorporated into the structure of CNN to enhance feature learning using spatial and channel attention mechanisms. Then, the final feature vector is formed by concatenating these features. This feature vector is provided to the classification layers to predict static sign gestures. The proposed approach is validated on two publicly available static Massey American Sign Language (ASL) and Indian Sign Language (ISL) databases. The model’s performance is evaluated using precision, recall, F1-score, and accuracy. Our proposed methodology achieved 99.22% and 99.79% accuracy for the ASL and ISL datasets. The acquired results signify the efficiency of the feature fusion and attention mechanism. Our network performed better in accuracy compared to the earlier studies.

show abstract

“…However, the study acknowledges that there is room for improving the accuracy of the model and highlights the limitation of limited dataset availability. In their research, the authors introduce "Sign Explainer [25]," a framework that combines explainable AI techniques with ensemble learning for sign language recognition. The framework achieves an impressive overall recognition rate of 98% and 92.60%.…”

Section: Deep Learning In Recognizing Sign Languagementioning

confidence: 99%

Robot Assist Sign Language Recognition for Hearing Impaired Persons Using Deep Learning

Khan,

Aslam,

Abid

et al. 2023

VAWKUM trans. comput. sci.

View full text Add to dashboard Cite

The adoption of Sign Language Communication (SLC) systems has become more significant in closing the interaction between the deaf society and the world of hearing people. In this study, researchers aim to contribute to this field by developing a system that helps sign language users communicate through BERT approaches based on deep learning frameworks as well as NLP. Accurate recognition of sign language is critical for SLC systems to work well. Deep learning models are effective in identifying sign language with high accuracy. This study aims to determine the most suitable DL model for identifying sign language and assess the impact of incorporating Natural Language Processing (NLP) techniques in generating frequent and accurate responses in SLC systems. The NLP model will be developed as an optimum return mechanism to generate frequent responses. This research includes testing three different deep learning models: MLP, CNN, and RestNet50v2 to recognize sign language gestures. Restnet50v2 outscored the other two approaches with a 0.97% perfection. As said earlier, the system also generates automated responses using the NLP BERT model, with an overall accuracy of 0.8% and a BLEU score of 0.83%. This method has a way to enhance interaction among the deaf community via the use of technology, opening new avenues for developing intelligent chatbots that can better understand nonverbal communication. Further research can be done to expand its functionality to recognize a broader range of sign language gestures and improve the user interface. Overall, this study demonstrates how technology can enhance the ways of people with deafness or hearing loss by addressing communication barriers.

show abstract

SignExplainer: An Explainable AI-Enabled Framework for Sign Language Recognition With Ensemble Learning

Cited by 16 publications

References 39 publications

Enhancing Fingerprint Liveness Detection Accuracy Using Deep Learning: A Comprehensive Study and Novel Approach

Enhancing Fingerprint Liveness Detection Accuracy Using Deep Learning: A Comprehensive Study and Novel Approach

Fusion of Attention-Based Convolution Neural Network and HOG Features for Static Sign Language Recognition

Robot Assist Sign Language Recognition for Hearing Impaired Persons Using Deep Learning

Contact Info

Product

Resources

About