Cross modal comprehension in ZARDOZ an English to sign-language translation system

Veale, Tony; Conway, Alan

doi:10.3115/1641417.1641450

Cited by 16 publications

(6 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Furthermore, virtual mouse systems need enhancement in precision and efficiency. Papers [10], [11], [12], [13], [14], [15], [16], [17] revolve around the above stated techniques which is explained in the Table I. Followed by Lalit Goyal et al [18] in 2016 constructed a synthetic dictionary that classified ISL words.…”

Section: Related Workmentioning

confidence: 99%

Harnessing AI to Generate Indian Sign Language from Natural Speech and Text for Digital Inclusion and Accessibility

Yadav,

Sharma,

Khanna

et al. 2024

IJACSA

View full text Add to dashboard Cite

Sign language is the fundamental mode of communication for those who are deaf and mute, as well as for individuals with hearing impairments. Regrettably, there has been a dearth of research on Indian Sign Language, primarily due to the lack of adequate grammar and regional variations in such language. Consequently, research in this area has been limited. The primary objective of our research is to develop a sophisticated speech/ text-to-Indian sign language conversion system that employs advanced 3D modeling techniques to display sign language motions. Our research is motivated by our desire to promote effective communication between hearing and hearingimpaired individuals in India. The proposed model integrates Automatic Speech Recognition (ASR) technology, which effectively transforms spoken words into text, and leverages 3D modeling techniques to generate corresponding sign language motions. We have conducted a comprehensive study of the grammar of Indian Sign Language, which includes identifying sentence structure and signs that represent the tense of the subject. It is noteworthy that the sentence structure of Indian Sign Language follows the Subject-Object-Verb sequence, in contrast to spoken language, which follows the Subject-Verb-Object structure. To enhance user experience as well as digital inclusion and accessibility, the research incorporates user-friendly and simple interfaces that allow individuals to interact effortlessly with the system intuitively. The model/ system is equipped to receive speech input through a microphone/ text and provide immediate feedback through 3D-modeled videos that display the generated sign language gestures and has achieved 99.2% accuracy. Our main goal is to promote digital inclusion and improve accessibility and enhance the user experience.

show abstract

Section: Related Workmentioning

confidence: 99%

Harnessing AI to Generate Indian Sign Language from Natural Speech and Text for Digital Inclusion and Accessibility

Yadav,

Sharma,

Khanna

et al. 2024

IJACSA

View full text Add to dashboard Cite

show abstract

“…The main advantage of this system is the efficient performance for processing semantic information. The contributors of this project aim to improve this system using intelligent linguistic technologies [59].…”

Section: Datasetsmentioning

confidence: 99%

All You Need In Sign Language Production

Rastgoo¹,

Kiani²,

Escalera³

et al. 2022

Preprint

View full text Add to dashboard Cite

Sign Language is the dominant form of communication language used in the deaf and hearing-impaired community. To make an easy and mutual communication between the hearing-impaired and the hearing communities, building a robust system capable of translating the spoken language into sign language and vice versa is fundamental. To this end, sign language recognition and production are two necessary parts for making such a two-way system. Sign language recognition and production need to cope with some critical challenges. In this survey, we review recent advances in Sign Language Production (SLP) and related areas using deep learning. To have more realistic perspectives to sign language, we present an introduction to the Deaf culture, Deaf centers, psychological perspective of sign language, the main differences between spoken language and sign language. Furthermore, we present the fundamental components of a bi-directional sign language translation system, discussing the main challenges in this area. Also, the backbone architectures and methods in SLP are briefly introduced and the proposed taxonomy on SLP is presented. Finally, a general framework for SLP and performance evaluation, and also a discussion on the recent developments, advantages, and limitations in SLP, commenting on possible lines for future research are presented.

show abstract

“…There have also been systems such as the ZARDOS translator developed by the Hitachi Dublin Laboratory [4], that uses blackboard architecture, an artificial intelligence approach in which common knowledge is loaded onto a common unit called the blackboard and updated by external agents who have access to specialist information. Once the English phrase is analyzed for semantic nature and structure, the sign synthesizer produces an animation of the signed translation using the DCL (Doll Control Language) animation software.…”

Section: B Zardosmentioning

confidence: 99%

A Translation System That Converts English Text to American Sign Language Enhanced with Deep Learning Modules

Lalitha¹,

Shetty²

2019

IJITEE

View full text Add to dashboard Cite

A recent surge in interest to create translation systems inclusive of sign languages is engendered by not only the rapid development of various approaches in the field of machine translation, but also the increased awareness of the struggles of the deaf community to comprehend written English. This paper describes the working of SILANT (SIgn LANguage Translator), a machine translation system that converts English to American Sign Language (ASL) using the principles of Natural Language Processing (NLP) and Deep Learning. The translation of English text is based on transformational rules which generates an intermediate representation which in turn spawns appropriate ASL animations. Although this kind of rule-based translation is notorious for being an accurate yet narrow approach, in this system, we broaden the scope of the translation using a synonym network and paraphrasing module which implements deep learning algorithms. In doing so, we are able to achieve both the accuracy of a rule-based approach and the scale of a deep learning one.

show abstract

Cross modal comprehension in ZARDOZ an English to sign-language translation system

Cited by 16 publications

References 6 publications

Harnessing AI to Generate Indian Sign Language from Natural Speech and Text for Digital Inclusion and Accessibility

Harnessing AI to Generate Indian Sign Language from Natural Speech and Text for Digital Inclusion and Accessibility

All You Need In Sign Language Production

A Translation System That Converts English Text to American Sign Language Enhanced with Deep Learning Modules

Contact Info

Product

Resources

About