Malaysian sign language dataset for automatic sign language recognition system

Karbasi, Mahdie; Zabidi, Azlee; Yassin, Ihsan Mohd; Ahmed, Waqas; Bhatti, Zeeshan

doi:10.4314/jfas.v9i4s.26

Cited by 10 publications

(8 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They trained the network with a learning rate of 0.05 using 225 samples and achieved 80.54% accuracy on 15 dynamic signs. In 2017, Karbasi, Zabidi, Yassin, Waqas, and Bhatti [27] demonstrated a dataset development for MSL consisting of alphabets and ten (10) dynamic signs using Microsoft Kinect.…”

Section: Relevant Past Studies On Malaysian Sign Languagementioning

confidence: 99%

Evaluating the Efficiency of CBAM-Resnet Using Malaysian Sign Language

Khan¹,

Wong²,

Ullah³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

The deaf-mutes population is constantly feeling helpless when others do not understand them and vice versa. To fill this gap, this study implements a CNN-based neural network, Convolutional Based Attention Module (CBAM), to recognise Malaysian Sign Language (MSL) in videos recognition. This study has created 2071 videos for 19 dynamic signs. Two different experiments were conducted for dynamic signs, using CBAM-3DResNet implementing 'Within Blocks' and 'Before Classifier' methods. Various metrics such as the accuracy, loss, precision, recall, F1-score, confusion matrix, and training time were recorded to evaluate the models' efficiency. Results showed that CBAM-ResNet models had good performances in videos recognition tasks, with recognition rates of over 90% with little variations. CBAM-ResNet 'Before Classifier' is more efficient than 'Within Blocks' models of CBAM-ResNet. All experiment results indicated the CBAM-ResNet 'Before Classifier' efficiency in recognising Malaysian Sign Language and its worth of future research.

show abstract

Section: Relevant Past Studies On Malaysian Sign Languagementioning

confidence: 99%

Evaluating the Efficiency of CBAM-Resnet Using Malaysian Sign Language

Khan¹,

Wong²,

Ullah³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

show abstract

“…They trained the network with a learning rate of 0.05 using 225 samples and achieved 80.54% accuracy on 15 dynamic signs. In 2017, Karbasi et al [ 30 ] demonstrated a dataset development for MSL consisting of alphabets and ten (10) dynamic signs using Microsoft Kinect. In 2019, Fahmi et al [ 31 ] proposed a hand signs translator system based on the fuzzy logic method.…”

Section: Literature Reviewmentioning

confidence: 99%

Intelligent Malaysian Sign Language Translation System Using Convolutional‐Based Attention Module with Residual Network

Khan

Khattak

Wong

et al. 2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

The deaf-mutes population always feels helpless when they are not understood by others and vice versa. This is a big humanitarian problem and needs localised solution. To solve this problem, this study implements a convolutional neural network (CNN), convolutional-based attention module (CBAM) to recognise Malaysian Sign Language (MSL) from images. Two different experiments were conducted for MSL signs, using CBAM-2DResNet (2-Dimensional Residual Network) implementing “Within Blocks” and “Before Classifier” methods. Various metrics such as the accuracy, loss, precision, recall, F1-score, confusion matrix, and training time are recorded to evaluate the models’ efficiency. The experimental results showed that CBAM-ResNet models achieved a good performance in MSL signs recognition tasks, with accuracy rates of over 90% through a little of variations. The CBAM-ResNet “Before Classifier” models are more efficient than “Within Blocks” CBAM-ResNet models. Thus, the best trained model of CBAM-2DResNet is chosen to develop a real-time sign recognition system for translating from sign language to text and from text to sign language in an easy way of communication between deaf-mutes and other people. All experiment results indicated that the “Before Classifier” of CBAMResNet models is more efficient in recognising MSL and it is worth for future research.

show abstract

“…Communication is a tedious task for people who have hearing and speech impairments. Hand gestures, which involve the movement of hands, are used as sign language for natural communication between ordinary people and deaf people, which is just like speech for vocal people [4,[6][7][8][9]. Nonetheless, sign languages differ by country and are used for a variety of purposes, including American Sign Language (ASL), British Sign Language (BSL), Japanese Sign Language [10,11], and Turkish Sign Language (TSL) [12].…”

Section: Introductionmentioning

confidence: 99%

Interpretation of Bahasa Isyarat Malaysia (BIM) Using SSD-MobileNet-V2 FPNLite and COCO mAP

et al. 2023

View full text Add to dashboard Cite

This research proposes a study on two-way communication between deaf/mute and normal people using an Android application. Despite advancements in technology, there is still a lack of mobile applications that facilitate two-way communication between deaf/mute and normal people, especially by using Bahasa Isyarat Malaysia (BIM). This project consists of three parts: First, we use BIM letters, which enables the recognition of BIM letters and BIM combined letters to form a word. In this part, a MobileNet pre-trained model is implemented to train the model with a total of 87,000 images for 29 classes, with a 10% test size and a 90% training size. The second part is BIM word hand gestures, which consists of five classes that are trained with the SSD-MobileNet-V2 FPNLite 320 × 320 pre-trained model with a speed of 22 s/frame rate and COCO mAP of 22.2, with a total of 500 images for all five classes and first-time training set to 2000 steps, while the second- and third-time training are set to 2500 steps. The third part is Android application development using Android Studio, which contains the features of the BIM letters and BIM word hand gestures, with the trained models converted into TensorFlow Lite. This feature also includes the conversion of speech to text, whereby this feature allows converting speech to text through the Android application. Thus, BIM letters obtain 99.75% accuracy after training the models, while BIM word hand gestures obtain 61.60% accuracy. The suggested system is validated as a result of these simulations and tests.

show abstract

Malaysian sign language dataset for automatic sign language recognition system

Cited by 10 publications

References 8 publications

Evaluating the Efficiency of CBAM-Resnet Using Malaysian Sign Language

Evaluating the Efficiency of CBAM-Resnet Using Malaysian Sign Language

Intelligent Malaysian Sign Language Translation System Using Convolutional‐Based Attention Module with Residual Network

Interpretation of Bahasa Isyarat Malaysia (BIM) Using SSD-MobileNet-V2 FPNLite and COCO mAP

Contact Info

Product

Resources

About