Design, development and field evaluation of a Spanish into sign language translation system

Abstract

This paper describes the design, development and field evaluation of a machine translation system from Spanish to Spanish Sign Language (LSE: Lengua de Signos Española). The developed system focuses on helping Deaf people when they want to renew their Driver's License. The system is made up of a speech recognizer (for decoding the spoken utterance into a word sequence), a natural language translator (for converting a word sequence into a sequence of signs belonging to the sign language), and a 3D avatar animation module (for playing back the signs). For the natural language translator, three technological approaches have been implemented and evaluated: an example-based strategy, a rule-based translation method and a statistical translator. For the final version, the implemented language translator combines all the alternatives into a hierarchical structure. This paper includes a detailed description of the field evaluation. This evaluation was carried out in the Local Traffic Office in Toledo involving real government employees and Deaf people. The evaluation includes objective measurements from the system and subjective information from questionnaires.The paper details the main problems found and a discussion on how to solve them (some of them specific for LSE).

A System to Generate SignWriting for Video Tracks Enhancing Accessibility of Deaf People

“…In order to measure these types of errors, next tests will include new metrics such as Sign Error Rate (SER). SER is the percentage of wrong signs in the translation output compared to a reference, which is provided by a human expert [14]. Besides, new recent instruments [26] [27] to evaluate usability and accessibility will be considered to assure an efficient evaluation of user experience in next experiments.…”

“…San Segundo et al [14] have developed a machine translation system from Spanish to Spanish Sign Language. The system uses speech recognition, which is automatically translated and interpreted by an avatar animation module.…”

“…Most of translators to Sign Language correspond to automatic translators focused on specific domains of activity, in which vocabulary and expected sentence constructions are quite limited, such as a hotel reception [16], a bus company [17], renewing the identity card or the driver's license [14], for example.…”

A rule-based translation from written Spanish to Spanish Sign Language glosses

“…To our knowledge, there are no published results on this system but it is available on the Internet. 5 The most complete work in Spanish-to-LSE translation to date is summarized in (San-Segundo et al, 2011). Their system focuses on helping Deaf people in a highly specific domain, the renewal of driving licenses.…”

Translating Italian to LIS in the Rail Stations

“…Among them: post office announcements (Cox et al, 2002), weather forecasting (Verlinden et al, 2001;), driver's license renewal (San-Segundo et al, 2012), and train announcements (Segouat and Braffort, 2009;Ebling, 2013). However, SLs still pose many challenges related to the specific linguistic features (e.g.…”