Real-time computer vision-based Bengali Sign Language recognition

Rahaman, Muhammad Aminur; Jasim, Mahmood; Ali, Md. Yousuf; Hasanuzzaman, Md.

doi:10.1109/iccitechn.2014.7073150

Cited by 71 publications

(25 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yasir et al [11] and Shanta et al [12] proposed a SIFT-based feature extraction followed by SVM classification and CNN respectively to recognize BdSL alphabets. Rahaman et al [13] proposed Haar-like feature-based classifier to recognize 36 alphabets of two-handed BdSL recognition. In another work, Rahaman et al [14] proposed a Bangla language modeling algorithm (BLMA) for 36 two-handed BdSL alphabet and 10 BdSL digits recognition.…”

Section: Literature Reviewmentioning

confidence: 99%

Design and Development of a Humanoid Robot for Sign Language Interpretation

et al. 2021

View full text Add to dashboard Cite

Purpose With an indisputable complexity of communication for hearing and speaking impaired people, most sign language recognition systems utilize virtual reality or onscreen robots. This paper presents the design and development of a special and low-cost humanoid robot that can perform as a sign language interpreter. To the best of our knowledge, this is the first endeavor to fabricate a humanoid robot for Bangla sign language (BdSL) and Medical signs interpretation. Methods Considering the plethora of design criteria and balancing between rigidity and flexibility 3D models of the robotics parts are designed and 3D printed ensuring cost efficiency. With the help of modern fabrication technology, the robot is developed and assembled with proper actuators and circuitry. An image dataset is built comprising 950 images for BdSL recognition and made publicly available. We utilized the Recurrent neural network (RNN) and Convolutional neural network (CNN) for deep learning model establishment and feature extraction from video and image data. Results The developed humanoid robot has 43 Degrees of freedom (DoF) which includes two 15 DoF hands. It can imitate 16 BdSL alphabets in sign language, can capture a video or image input in real-time from the user, and recognize 10 medical signs and 38 alphabets of BdSL. The learning model for video-based medical sign recognition achieved 87.5% test accuracy. Image-based Bangla sign language recognition achieved an overall test accuracy of 98.19% in our dataset and 93.8% in another available dataset. Conclusion Compared to the state-of-the-art robotic systems for sign language interpretation, our approach has achieved higher kinematic characteristics, remarkable results in sign recognition, and impressive competency in sign imitation; all at almost 10 times lower cost than the state-of-the-art systems. The results are evidence that our approach is efficient and suitable in helping hearing and speaking impaired people. Moreover, this work initiates a research scope that can be further extended for creating equal opportunities for the hearing and speaking impaired community.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Design and Development of a Humanoid Robot for Sign Language Interpretation

et al. 2021

View full text Add to dashboard Cite

show abstract

“…In paper [19], the authors developed a BSL recognition system.The system is basially a real time computer vision based system where hand area is detected from the input image using the classifier namely Haar-like feature-based cascaded classifier [20].Using K-Nearest Neighbor classifier, hand sign is extracted by the proposed system from Hue and Saturation values of HSV color model where HSV stands for Hue, Saturation, and Value. HSV color model is frequently used instead of the RGB color model in application programs such as graphics and paint programs.…”

Section: Related Workmentioning

confidence: 99%

Automated Bangla sign language translation system for alphabets by means of MobileNet

et al. 2020

View full text Add to dashboard Cite

Individuals with hearing and speaking impairment communicate using sign language. The movement of hand, body and expressions of face are the means by which the people, who are unable to hear and speak, can communicate. Bangla sign alphabets are formed with one or two hand movements. There are some features which differentiates the signs. To detect and recognize the signs, analyzing its shape and comparing its features is necessary. This paper aims to propose a model and build a computer systemthat can recognize Bangla Sign Lanugage alphabets and translate them to corresponding Bangla letters by means of deep convolutional neural network (CNN). CNN has been introduced in this model in form of a pre-trained model called "MobileNet" which produced an average accuracy of 95.71% in recognizing 36 Bangla Sign Language alphabets.

show abstract

“…In the last few years, many hand-gesture recognition techniques have been proposed and many systems have been developed for different sign languages all over the world. Rahman et al [2] work on hand-sign recognition for Bengali Sign Language to detect some of the Bengali Vowels and Consonants. They have used Haar-like features and K-Nearest Neighbors Classifier to build their model.…”

Section: Literature Surveymentioning

confidence: 99%

Hand Sign to Bangla Speech: A Deep Learning in Vision Based System for Recognizing Hand Sign Digits and Generating Bangla Speech

et al. 2019

View full text Add to dashboard Cite

Recent advancements in the field of computer vision with the help of deep neural networks have led us to explore and develop many existing challenges that were once unattended due to the lack of necessary technologies. Hand Sign/Gesture Recognition is one of the significant areas where the deep neural network is making a substantial impact. In the last few years, a large number of researches has been conducted to recognize hand signs and hand gestures, which we aim to extend to our mother-tongue, Bangla (also known as Bengali). The primary goal of our work is to make an automated tool to aid the people who are unable to speak. We developed a system that automatically detects hand sign based digits and speaks out the result in Bangla language. According to the report of the World Health Organization (WHO), 15% of people in the world live with some kind of disabilities. Among them, individuals with communication impairment such as speech disabilities experience substantial barrier in social interaction. The proposed system can be invaluable to mitigate such a barrier. The core of the system is built with a deep learning model which is based on convolutional neural networks (CNN). The model classifies hand sign based digits with 92% accuracy over validation data which ensures it a highly trustworthy system. Upon classification of the digits, the resulting output is fed to the text to speech engine and the translator unit eventually which generates audio output in Bangla language. A web application to demonstrate our tool is available at http://bit.ly/signdigits2banglaspeech.

show abstract

Real-time computer vision-based Bengali Sign Language recognition

Cited by 71 publications

References 3 publications

Design and Development of a Humanoid Robot for Sign Language Interpretation

Design and Development of a Humanoid Robot for Sign Language Interpretation

Automated Bangla sign language translation system for alphabets by means of MobileNet

Hand Sign to Bangla Speech: A Deep Learning in Vision Based System for Recognizing Hand Sign Digits and Generating Bangla Speech

Contact Info

Product

Resources

About