Two-handed hand gesture recognition for Bangla sign language using LDA and ANN

Individuals with hearing and speaking impairment communicate using sign language. The movement of hand, body and expressions of face are the means by which the people, who are unable to hear and speak, can communicate. Bangla sign alphabets are formed with one or two hand movements. There are some features which differentiates the signs. To detect and recognize the signs, analyzing its shape and comparing its features is necessary. This paper aims to propose a model and build a computer systemthat can recognize Bangla Sign Lanugage alphabets and translate them to corresponding Bangla letters by means of deep convolutional neural network (CNN). CNN has been introduced in this model in form of a pre-trained model called "MobileNet" which produced an average accuracy of 95.71% in recognizing 36 Bangla Sign Language alphabets.

show abstract

“…ANN classifier with LDA produced an excellent result. The result is much better than ANN with PCA or raw image [13].…”

Section: Related Workmentioning

confidence: 93%

Automated Bangla sign language translation system for alphabets by means of MobileNet

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Both Jarman et al [15] and Ahmed and Akhand [16] implemented feed-forward ANN to recognize one-handed BdSL alphabets. Yasir and Khan [17] also deployed ANN for 15 two-handed BdSL alphabets recognition and analyzed performance between Principal Component Analysis and Linear Discriminant Analysis. Afjal et al [18] utilized HOG features and KNN classifier in the recognition of 38 one-handed BdSL alphabets and 10 digits.…”

Section: Literature Reviewmentioning

confidence: 99%

Design and Development of a Humanoid Robot for Sign Language Interpretation

et al. 2021

View full text Add to dashboard Cite

Purpose With an indisputable complexity of communication for hearing and speaking impaired people, most sign language recognition systems utilize virtual reality or onscreen robots. This paper presents the design and development of a special and low-cost humanoid robot that can perform as a sign language interpreter. To the best of our knowledge, this is the first endeavor to fabricate a humanoid robot for Bangla sign language (BdSL) and Medical signs interpretation. Methods Considering the plethora of design criteria and balancing between rigidity and flexibility 3D models of the robotics parts are designed and 3D printed ensuring cost efficiency. With the help of modern fabrication technology, the robot is developed and assembled with proper actuators and circuitry. An image dataset is built comprising 950 images for BdSL recognition and made publicly available. We utilized the Recurrent neural network (RNN) and Convolutional neural network (CNN) for deep learning model establishment and feature extraction from video and image data. Results The developed humanoid robot has 43 Degrees of freedom (DoF) which includes two 15 DoF hands. It can imitate 16 BdSL alphabets in sign language, can capture a video or image input in real-time from the user, and recognize 10 medical signs and 38 alphabets of BdSL. The learning model for video-based medical sign recognition achieved 87.5% test accuracy. Image-based Bangla sign language recognition achieved an overall test accuracy of 98.19% in our dataset and 93.8% in another available dataset. Conclusion Compared to the state-of-the-art robotic systems for sign language interpretation, our approach has achieved higher kinematic characteristics, remarkable results in sign recognition, and impressive competency in sign imitation; all at almost 10 times lower cost than the state-of-the-art systems. The results are evidence that our approach is efficient and suitable in helping hearing and speaking impaired people. Moreover, this work initiates a research scope that can be further extended for creating equal opportunities for the hearing and speaking impaired community.

show abstract

“…They reported an accuracy of 97.7% with their method, which involved converting RGB images to the HSV color space, extracting features using a bank of Gabor Filters, and using Kernel PCA for dimensionality reduction. Yasir et al [25], in their work, trained on a relatively small dataset of 330 RGB images using PCA and Linear Discriminant Analysis (LDA) to minimize intra-class scatter and maximize interclass distance. Yasir et al [26] used SIFT features on a vocabulary dataset of only 90 images.…”

Section: A Machine Learning Techniquesmentioning

confidence: 99%

Generalization of Bangla Sign Language Recognition Using Angular Loss Functions

Youme¹,

Chowdhury

Ahamed

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Sign Language provides the means of conveying messages for deaf and mute people. Effective communication with the masses is a great challenge for the deaf and mute community, as Sign Language is not commonly understood. Many researchers have done numerous works in foreign language datasets like English, French, Japanese, etc. However, for Bangla, one of the most widely spoken languages, much significant work has not been done yet. Most of the works on Bangla Sign Language are executed on small datasets and report satisfactory performance. However, when small datasets are evaluated from the perspective of generalizability, particularly when using deep learning based solutions, these models fail to reproduce to the same performance. Therefore, this paper poses inter-dataset evaluation as the main evaluation criteria and evaluates several deep learning based models. This evaluation is done for Bangla by leveraging two popular datasets of Bangla Sign Language. Unsurprisingly, the inter-dataset performance is inferior, and several approaches to improve are explored and documented, including the use of angular margin based loss functions. The results demonstrate the importance of such an evaluation and also show that one of the proposed approaches shows promising performance, albeit with significant room for improvement. This raises the need for a standardized dataset to overcome this issue of generalization for real-life applications, as well as the need to encourage future works to concentrate on challenging evaluations instead of pursuing deceptively good intra-dataset performance.

show abstract

Two-handed hand gesture recognition for Bangla sign language using LDA and ANN

Cited by 25 publications

References 6 publications

Automated Bangla sign language translation system for alphabets by means of MobileNet

Automated Bangla sign language translation system for alphabets by means of MobileNet

Design and Development of a Humanoid Robot for Sign Language Interpretation

Generalization of Bangla Sign Language Recognition Using Angular Loss Functions

Contact Info

Product

Resources

About