The Development and Evaluation of a Speech-to-Sign Translation System to Assist Transactions

Journal of Visual Languages & Computing

2012

Esta es la versión de autor de la comunicación de congreso publicada en: This is an author produced version of a paper published in: AbstractThis work presents a new approach to the synthesis of Spanish Sign Language (LSE). Its main contributions are the use of a centralized relational database for storing sign descriptions, the proposal of a new input notation and a new avatar design, the skeleton structure of which improves the synthesis process.The relational database facilitates a highly detailed phonologic description of the signs that, include parameter synchronization and timing. The centralized database approach has been introduced to allow the representation of each sign to be validated by the LSE national institution, FCNSE.The input notation, designated HLSML, presents multiple levels of abstraction compared with current input notations. Redesigned input notation is used to simplify the description and the manual definition of LSE messages.Synthetic messages obtained using our approach have been evaluated by deaf users; in this evaluation a maximum recognition rate of 98.5% was obtained for isolated signs and a recognition rate of 95% was achieved for signed sentences.

Section: Discussionsupporting

confidence: 71%

Section: Comparison With Other Workmentioning

confidence: 99%

Spanish Sign Language synthesis system

Journal of Visual Languages & Computing

2012

“…The sign recognition rates of deaf user have been reported to be lower than the results obtained when the users who evaluated the system were SL interpreters (López-Colino and Colás, 2011). The acceptance of synthetic signed messages is also lower when evaluated by deaf users (Cox et al, 2003). Deaf users provide the baseline of the system.…”

Section: Discussionmentioning

confidence: 99%

“…The validity of our claims was tested through user evaluations, with respect to SL synthesis (Cox et al, 2003;Huenerfauth et al, 2008;Huenerfauth, 2008) and visual prosody (Hartmann et al, 2005a); user evaluation is the only approach to test a system or a hypothesis of this kind. The main difference 15 between the two topics is the number of users who participate in the experiments.…”

Section: Methodsmentioning

confidence: 99%

Synthesizing mood-affected signed messages: Modifications to the parametric synthesis

International Journal of Human-Computer Studies

2012

This paper describes the first approach in synthesizing mood-affected signed contents. The research focuses on the modifications applied to a parametric sign language synthesizer (based on phonetic descriptions of the signs). We propose some modifications that will allow for the synthesis of different perceived frames of mind within synthetic signed messages. Three of these proposals focus on modifications to three different signs' phonologic parameters (the hand shape, the movement and the non-hand parameter). The other two proposals focus on the temporal aspect of the synthesis (sign speed and transition duration) and the representation of muscular tension through inverse kinematics procedures. These resulting variations have been evaluated by Spanish deaf signers, who have concluded that our system can generate the same signed message with three different frames of mind, which are correctly identified by Spanish Sign Language signers.

“…The recognition rate of ViSiCAST system is 81% for isolated signs and 61% for complete phrases [5]. The experiments allowed three views of each video.…”

Section: Synthetic Sl Messages Evaluationmentioning

confidence: 99%

Hybrid paradigm for Spanish Sign Language synthesis

2011

Univ Access Inf Soc

This work presents a hybrid approach to sign language synthesis. This approach allows hand-tuning the phonetic description of the signs, focusing on the time aspect of the sign. Therefore, we keep the capacity of performing morphonological operations, like the notation-based approaches and improving the synthetic signing performance, like the hand-tuned animations approach.Our approach simplifies input message description using a new high level notation and storing sign phonetic descriptions in a relational database. The relational database allows more flexible sign phonetic descriptions; it also allows describing sign timing and the synchronization between sign phonemes. The new notation, named HLSML, focuses on message description; it is a gloss-based notation. HLSML introduces several tags that allow modifying the signs in the message defining dialect and mood variations (both defined in the relational database) and message timing (transition durations and pauses). We also propose a new avatar design that simplifies the development of the synthesizer and avoids any interference in the independence of the sign language phonemes during the animation.The obtained results show an increase of the sign recognition rate compared to other approaches. This improvement is based on the active role that the sign language experts have in the description of signs, allowed by the flexibility of the sign storage approach. The approach will simplify the description of synthesizable signed messages so the creation of multimedia signed contents will be easier.