Design of a Robotic Speech-to-Sign Language Transliterating System

Zapata, Erwin Daniel López; Trinidad, Maykol J. Campos; Salazar-Arévalo, R.; Carpio, J. Acostupa-Del

doi:10.1007/978-3-319-67567-1_26

Cited by 1 publication

(1 citation statement)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A major reason for the communication barriers is that majority of hearing impaired users are illiterate [11,27,69,80,85]. To overcome the communication gap between the hearing impaired and hearing people there is an urgent need of translator(s) [62,77]. Human translators are available only in major public areas or people hire translators as per their needs.…”

Section: User Requirementsmentioning

confidence: 99%

An automatic machine translation system for multi-lingual speech to Indian sign language

Dhanjal

Singh

2021

Multimed Tools Appl

View full text Add to dashboard Cite

Sign language (SL) is the best suited communication medium for hearing impaired people. Even with the advancement of technology, there is a communication gap between the hearing impaired and hearing people. The aim of this research work is to bridge this gap by developing an automatic system that translates the speech to Indian Sign Language using Avatar (SISLA). The whole system works in three phases: (i) The first phase includes the speech recognition (SR) of isolated words for English, Hindi and Punjabi in speaker independent environment (ii) The second phase translates the source language into Indian Sign Language (ISL) (iii) HamNoSys based 3D avatar represents the ISL gestures. The four major implementation modules for SISLA include: requirement analysis, data collection, technical development and evaluation. The multi-lingual feature makes the system more efficient. The training and testing speech sample files for English (12,660, 4218), Hindi (12,610, 4211) and Punjabi (12,600, 4193) have been used to train and test the SR models. Empirical results of automatic machine translation show that the proposed trained models have achieved the minimum accuracy of 91%, 89% and 89% for English, Punjabi and Hindi respectively. Sign language experts have also been used to evaluate the sign error rate through feedback. Future directions to enhance the proposed system using non-manual SL features along with the sentence level translation has been suggested. Usability testing based on survey results confirm that the proposed SISLA system is suitable for education as well as communication purpose for hearing impaired people.

show abstract