Tessa, a system to aid communication with deaf people

Cox, Stephen J.; Lincoln, Michael J.; Tryggvason, Judy; Nakisa, Melanie; Wells, Mark B.; Tutt, Marcus; Abbott, Sanja

doi:10.1145/638249.638287

Cited by 128 publications

(55 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The second technique joins motion captured animation sequences of signs to animate 3D avatars (Awad, Courty, Duarte, Le Naour & Gibet, 2009;Cox et al, 2002). A markup language SiGML (Elliott, Glauert, Kennaway, & Parsons, 2001) is used to define signs from a common set of motion captured hand postures and their transitions in the project "E-sign" (eSIGN project, 2002) that went on to translate government web pages in the British, Dutch and German sign languages.…”

Section: Techniques For Sign Language Synthesis and Animationmentioning

confidence: 99%

Computer Interpreter for Translating Written Sinhala to Sinhala Sign Language

Punchimudiyanse

Meegama

2017

OUSL Jnl

View full text Add to dashboard Cite

Sinhala Sign Language (SSL) is the preferred medium of communication amongst the deaf community of Sinhala. A conversational sign in SSL represents a word or phrase in Sinhala whereas a fingerspelling sign represents a character in the Sinhala alphabet. A word in Sinhala has different word forms based on tense and case structure of the Sinhala language. In the early stages of SSL evolution, a single SSL sign represented a base Sinhala word and all its word forms. This technique has deprived essential word form information to a deaf person. The SSL research community has come up with a technique of showing few characters of the expected word form in fingerspelling as prefix and/or suffixes with the sign of the base word to solve the problem. This research paper presents a database driven translator, which implements the said technique by translating written Sinhala sentences into SSL sequences. It gives the opportunity to the SSL research community to popularize this translation technique among SSL users. Moreover, a 3D avatar built earlier for animating spoken Sinhala in SSL is used to animate the translated SSL sequence. A test sample of 100 sentences and a word form database of 500+ entries to match 50 conversational SSL signs (30 nouns and 20 verbs) are formed to test the translator. The database driven search mechanism provided 100% successful translator performance while having 70.83% average sentence identification rate for six SSL users including three professional SSL interpreters.

show abstract

Section: Techniques For Sign Language Synthesis and Animationmentioning

confidence: 99%

Computer Interpreter for Translating Written Sinhala to Sinhala Sign Language

Punchimudiyanse

Meegama

2017

OUSL Jnl

View full text Add to dashboard Cite

show abstract

“…Such efforts have led to the development of many versions of SL, almost as many as the number of Arabic-speaking countries, owing to the varying heritage, culture and dialect with country; however, the same sign alphabets are used [21]. Since the Arab world has a very similar culture, heritage and sign alphabets, efforts are in progress to unify Arabic SL.…”

Section: Arabic Sign Languagementioning

confidence: 99%

Towards Empowering Hearing Impaired Students' Skills in Computing and Technology

Esam¹,

Abbas²,

Krause³

2017

ijacsa

View full text Add to dashboard Cite

Abstract-Studies have shown that deaf and hearing-impaired students have many difficulties in learning applied disciplines such as Medicine, Engineering, and Computer Programming. This study aims to investigate the readiness of deaf students to pursue higher education in applied sciences, more specifically in computer science. This involves investigating their capabilities in computer skills and applications. Computer programming is an integral component in the technological field that can facilitate the development of further scientific advances. Devising a manner of teaching the deaf and hearing-impaired population will give them an opportunity to contribute to the technology sector. This would allow these students to join the scientific world when otherwise; they are generally unable to participate because of the limitations they encounter. The study showed that deaf students in Jeddah are eager to continue their higher education and that a large percentage of these students are keen on studying computer science, particularly if they are provided with the right tools.

show abstract

“…In the UK, an experimental system called Text and Sign Support Assistant (TESSA) was developed to assist with transactions between a Deaf person and a clerk in a post office by translating the clerk's speech into British Sign Language (BSL) and the Deaf person's signs to text/sound [8]. To generate the signs needed for the TESSA system, the signs of the native signer are first captured as motion data via sensors affixed to the body of the signer.…”

Section: Assistive Applications For Deaf Usersmentioning

confidence: 99%

“…Automated sign language systems have a limited translation accuracy of merely 61% [10,20] which is unacceptable for pharmaceutical use. Moreover, using these systems require high-end smart phones or expensive computers [8,9,17]. We wish to minimize the cost of a DEV solution by using affordable mid-range technology that will soon become tomorrow's low-end.…”

Section: Introductionmentioning

confidence: 99%

Improving usability and correctness of a mobile tool to help a deaf person with pharmaceutical instruction

Motlhabi

Tucker

Parker

et al. 2013

Proceedings of the 4th Annual Symposium on Computing for Development

View full text Add to dashboard Cite

The computing for development community knows how to build user interfaces using qualitative methods for text illiterate users, especially on mobile devices. However, little work has been done specifically targeting Deaf users in developing regions who cannot access voice or text. This paper describes a multi-disciplinary collaboration towards iterative development of a mobile communication tool to support a Deaf person in understanding usage directions for medication dispensed at a pharmacy. We are improving usability and correctness of the user interface. The tool translates medicine instruction given in English text to Sign Language videos, which are relayed to a Deaf user on a mobile phone. Communication between pharmacists and Deaf patients were studied to extract relevant exchanges between the two users. We incorporated the common elements of these dialogues to represent content in a verifiable manner to ensure that the mobile tool relays the correct information to the Deaf user. Instructions are made available for a Deaf patient in signed language videos on a mobile device. A pharmacy setup was created to conduct trials of the tool with groups of end users, in order to collect usability data with recorded participant observation, questionnaires and focus group discussions. Subsequently, pre-recorded sign language videos, stored on a phone's memory card, were tested for correctness. Results of these two activities are presented and discussed in this paper, in preparation for a future trial in an actual pharmacy.

show abstract

Tessa, a system to aid communication with deaf people

Cited by 128 publications

References 9 publications

Computer Interpreter for Translating Written Sinhala to Sinhala Sign Language

Computer Interpreter for Translating Written Sinhala to Sinhala Sign Language

Towards Empowering Hearing Impaired Students' Skills in Computing and Technology

Improving usability and correctness of a mobile tool to help a deaf person with pharmaceutical instruction

Contact Info

Product

Resources

About