Deaf talk using 3D animated sign language: A sign language interpreter using Microsoft's kinect v2

Mateen, Abdul; Idrees, Mujtaba; Abideen, Zain Ul; Mumtaz, Rafia; Khalique, Sana

doi:10.1109/sai.2016.7556002

Cited by 39 publications

(7 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Research work offers the dual-mode of communication using Microsoft Kinect v2 device for hearing impaired people. Sign language to speech and speech to sign language with a 3D animated avatar along with the subtitles have been proposed [2]. State-of-the-art speech technology for live subtitle system has been designed for public service to be broadcasted in the Czech Republic Television [89].…”

Section: Related Workmentioning

confidence: 99%

An automatic machine translation system for multi-lingual speech to Indian sign language

Dhanjal

Singh

2021

Multimed Tools Appl

View full text Add to dashboard Cite

Sign language (SL) is the best suited communication medium for hearing impaired people. Even with the advancement of technology, there is a communication gap between the hearing impaired and hearing people. The aim of this research work is to bridge this gap by developing an automatic system that translates the speech to Indian Sign Language using Avatar (SISLA). The whole system works in three phases: (i) The first phase includes the speech recognition (SR) of isolated words for English, Hindi and Punjabi in speaker independent environment (ii) The second phase translates the source language into Indian Sign Language (ISL) (iii) HamNoSys based 3D avatar represents the ISL gestures. The four major implementation modules for SISLA include: requirement analysis, data collection, technical development and evaluation. The multi-lingual feature makes the system more efficient. The training and testing speech sample files for English (12,660, 4218), Hindi (12,610, 4211) and Punjabi (12,600, 4193) have been used to train and test the SR models. Empirical results of automatic machine translation show that the proposed trained models have achieved the minimum accuracy of 91%, 89% and 89% for English, Punjabi and Hindi respectively. Sign language experts have also been used to evaluate the sign error rate through feedback. Future directions to enhance the proposed system using non-manual SL features along with the sentence level translation has been suggested. Usability testing based on survey results confirm that the proposed SISLA system is suitable for education as well as communication purpose for hearing impaired people.

show abstract

Section: Related Workmentioning

confidence: 99%

An automatic machine translation system for multi-lingual speech to Indian sign language

Dhanjal

Singh

2021

Multimed Tools Appl

View full text Add to dashboard Cite

show abstract

“…World health organization has revealed that over 5% of the world population (430 million people) require rehabilitation to address their 'disabling' hearing loss, it was also mentioned that by the year 2050, over 700 million people will have a disabling hearing loss [1]. In 2020, it was stated by the World Federation of the Deaf that there are around 72 million people worldwide who uses sign language for communication [2]. According to the World health organization, approximately 9% of the Sri Lankan population has loss of hearing [3] and from a study conducted by the Sri Lanka Federation of the Deaf, more than 300,000 people are deaf.…”

Section: Introductionmentioning

confidence: 99%

Mask Region-Based Convolutional Neural Networks (R-CNN) for Sinhala Sign Language to Text Conversion

Sewwantha¹,

Ginige²

2021

Advances in Machine Learning, Data Mining and Computing

View full text Add to dashboard Cite

Sign Language is the use of various gestures and symbols for communication. It is mainly used by disabled people with communication difficulties due to their speech or hearing impediments. Due to the lack of knowledge on sign language, natural language speakers like us, are not able to communicate with such people. As a result, a communication gap is created between sign language users and natural language speakers. It should also be noted that sign language differs from country to country. With American sign language being the most commonly used, in Sri Lanka, we use Sri Lankan/Sinhala sign language. In this research, the authors propose a mobile solution using a Region Based Convolutional Neural Network for object detection to reduce the communication gap between the sign users and language speakers by identifying and interpreting Sinhala sign language to Sinhala text using Natural Language Processing (NLP). The system is able to identify and interpret still gesture signs in real-time using the trained model. The proposed solution uses object detection for the identification of the signs.

show abstract

“…They have applied a fuzzy C-means clustering algorithms and obtained 75% accuracy in gesture labelling. Ahmed et al proposed a 3D animated sign language recognition system which is named as DeafTalk [3]. This system works as an interpreter as well as translator making it a convenient medium for dual mode sign language recognition system.…”

Section: Introductionmentioning

confidence: 99%

Automatic Sign Language Identification Using Convolutional Neural Network

Mukherjee

Dhar

Obaidullah

et al. 2021

Advances in Intelligent Systems and Computing

View full text Add to dashboard Cite

Sign language is the mode of communication for such people who are not blessed with the gift of hearing and speech. It involves the use of hands and facial expressions, Understanding sign language is difficult for people with no prior knowledge. This leads to a communication gap for such people. An automated sign language identification system can aid in reducing this gap. Here, a system is presented to automatically identify sign language. Experiments were performed on the publicly available sign language MNIST dataset. The highest accuracy of 99.98% was obtained in this study with a convolutional neural network-based approach.

show abstract

Deaf talk using 3D animated sign language: A sign language interpreter using Microsoft's kinect v2

Cited by 39 publications

References 8 publications

An automatic machine translation system for multi-lingual speech to Indian sign language

An automatic machine translation system for multi-lingual speech to Indian sign language

Mask Region-Based Convolutional Neural Networks (R-CNN) for Sinhala Sign Language to Text Conversion

Automatic Sign Language Identification Using Convolutional Neural Network

Contact Info

Product

Resources

About