Czech Text-to-Sign Speech Synthesizer

Abstract

Abstract. Recent research progress in developing the Czech -Sign Speech synthesizer is presented. The current goal is to improve the system for automatic synthesis to produce accurate synthesis of the Sign Speech. The synthesis system converts written text to an animation of an artificial human model. This includes translation of text to sign phrases and its conversion to the animation of an avatar. The animation is composed of movements and deformations of segments of hands, a head and also a face. The system has been evaluated by two initial perceptual tests. The perceptual tests indicate that the designed synthesis system is capable of producing intelligible Sign Speech.

Automatic fingersign-to-speech translation system

“…The second type of systems generally rely on parametric models and scripts. These synthesis systems decode input such as special designed XML documents, HamNoSys or SignWriting notations to sequences of scripted animation commands: eSIGN [13], MUSSLAP [29] projects, the ILSP [17] and EMBR [21] systems, and synthetic corpus of American Sign Language (ASL) [49]. A system that supports the translation from Italian to Italian Sign Language (LIS) and generates avatar animation by combined approach is being designed [38].…”

“…We consider the data-driven approach for phonetic transcriptions of signs [32] and the rule based approach for HamNoSys (HNS-Hamburg Notation System) notation [29]. An algorithm automatically transferring HNS notation to control trajectories is the most important part of the sign speech synthesis system.…”

Correlation analysis of facial features and sign gestures

“…To enable the communication between deaf and hearing people systems of SL recognition and synthesis are developed [1], [2], [3]. SL is composed of manual and non-manual component [4].…”

“…In SL recognition they are combined to achieve better recognition rates [1]. In SL synthesis an avatar performs the signs by synthesizing both components [3]. In this paper we analyze the correlation between the components of SL for the purpose of SL synthesis.…”

“…The idea is to render animation of signs representing the translated text (or possibly speech). Current systems create the animation of mouth movements which correspond to the phonemes of the spoken language [3], [5]. It is known that native signers do not produce the mouthing in such way.…”