Linguistic modelling and language-processing technologies for Avatar-based sign language presentation

Rb, Elliott; Glauert, John; Kennaway, Richard; Marshall, I. Howard; Sáfár, Éva

doi:10.1007/s10209-007-0102-z

Cited by 114 publications

(67 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Several projects have focused on the synthesis of animations of virtual humans performing sign language [3,5,12,16] (and surveys in [12,13]). For example, several years of European projects have contributed to the eSIGN project, which creates technologies for content developers to build sign databases in a symbolic notation, assemble scripts of signing performance for use on accessible web pages, and allow viewers to see animations on their web browser [13].…”

Section: Motivations and Related Workmentioning

confidence: 99%

“…Content developers may script individual signs while observing the timing of human signers in video [13] or use motion-capture technology to record individual signs from humans directly [3], but this addresses the timing of isolated signs (not sentences). Also, many sign language animation systems give the viewer the ability to adjust a dial that modifies the speed of the performance [3,13]; however, section 2 will discuss how the speed of an ASL performance is more complex than a single speed value. Previous sign language animation research has not examined how the timeduration of a sign is affected by its surrounding linguistic context (what other signs occur in a sentence or in a performance) nor how pauses should be placed in an animation to mimic how human signers tend to pause at natural linguistic boundaries.…”

Section: Motivations and Related Workmentioning

confidence: 99%

“…SignSmith allows the content developer to manually specify pauses to occur, and content-scripting tools from the eSIGN project give similar control over speed, timing, and pauses [13]. Animations from generation or MT projects do tend to include pauses between sentences [3,5,12], but a principled linguistic way to select where to insert pauses into a sign language animation has not been described previously in the literature.…”

Section: Motivations and Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Evaluation of a psycholinguistically motivated timing model for animations of american sign language

Huenerfauth

2008

Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility

View full text Add to dashboard Cite

Using results in the psycholinguistics literature on the speed and timing of American Sign Language (ASL), we built algorithms to calculate the time-duration of signs and the location/length of pauses during an ASL animation. We conducted a study in which native ASL signers evaluated the ASL animations processed by our algorithms, and we found that: (1) adding linguistically motivated pauses and variations in sign-durations improved signers' performance on a comprehension task and (2) these animations were rated as more understandable by ASL signers. Categories and Subject Descriptors General TermsAlgorithms, Design, Experimentation, Human Factors. KeywordsAmerican Sign Language, Animation, Natural Language Generation, Evaluation, Accessibility Technology for the Deaf. MOTIVATIONS AND RELATED WORKAmerican Sign Language (ASL) is a full natural language; signers use their hands, facial expression, eye gaze, head tilt, and body posture to convey linguistic meaning. ASL is not just a manual presentation of an English sentence; it has its own word-order, syntactic constructions, and vocabulary of signs (which may not have one-to-one equivalence with English words). ASL is used as a primary means of communication for about one half million people in the United States [15]. Because of the differences between English and ASL, it is possible to have ASL fluency yet significant difficulty reading English. In fact, a majority of deaf 18-year-olds in the United States have an English reading level below average 10-year-old hearing students [10]. Unfortunately, websites and other written-English information sources can also pose a challenge for deaf adults with low literacy skills. Software that can display computer-generated animations of American Sign Language can make more information, communication, and services accessible to these users. These ASL animations may be scripted by a content developer [13,19] or generated by Englishto-ASL automatic machine translation software [2,12,14,17].ASL signing conveys information at the same rate as spoken English [1], but the average speed at which most adults can read English text is much faster than the speed of spoken English audio. For deaf adults with low literacy to access information at a speed comparable to English-reading adults, ASL animations would need to be displayed quickly (while maintaining their understandability). The literature contains a range of values for "normal" signing speed: from 1.5 to 2.37 signs per second [1,6]. When ASL videos are played faster than 2.5 times normal, viewers' comprehension of the video drops significantly [4].Studies of English speech have shown a link between the presence of linguistic pauses and comprehension of the audio. One study increased the speed of an English speech recording, and researchers later inserted pauses at linguistically appropriate locations (between sentences, between clauses, between phrases) [21]. The pauses improved listeners' comprehension. This benefit only arose if pauses were at linguistically appro...

show abstract

Section: Motivations and Related Workmentioning

confidence: 99%

Section: Motivations and Related Workmentioning

confidence: 99%

Section: Motivations and Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of a psycholinguistically motivated timing model for animations of american sign language

Huenerfauth

2008

Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility

View full text Add to dashboard Cite

show abstract

“…The most comprehensive existing sign language animation system based on synthesis from form notation is undoubtedly JASigning (Elliott et al, 2008;Jennings et al, 2010), a distant descendant of the avatar system used in TESSA which was further developed over the course of the eS-IGN and DictaSign European Framework projects. JASigning performs synthesis from SiGML (Elliott et al, 2000), an XML-based representation of the physical form of signs based on the wellunderstood Hamburg Notation System for Sign Languages (HamNoSys) (Prillwitz et al, 1989).…”

Section: Background and Related Workmentioning

confidence: 99%

An Open Web Platform for Rule-Based Speech-to-Sign Translation

Rayner¹,

Bouillon²,

Ebling³

et al. 2016

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

View full text Add to dashboard Cite

We present an open web platform for developing, compiling, and running rulebased speech to sign language translation applications. Speech recognition is performed using the Nuance Recognizer 10.2 toolkit, and signed output, including both manual and non-manual components, is rendered using the JASigning avatar system. The platform is designed to make the component technologies readily accessible to sign language experts who are not necessarily computer scientists. Translation grammars are written in a version of Synchronous Context-Free Grammar adapted to the peculiarities of sign language. All processing is carried out on a remote server, with content uploaded and accessed through a web interface. Initial experiences show that simple translation grammars can be implemented on a time-scale of a few hours to a few days and produce signed output readily comprehensible to Deaf informants. Overall, the platform drastically lowers the barrier to entry for researchers interested in building applications that generate high-quality signed language.

show abstract

“…Notations such as HamNoSys [5,6] and SignWriting [22] are graphic representations of SL and have equivalent computer-friendly versions, SiGML [23] for the HamNoSys notation and SWML [24] for the SignWriting version. In these projects [25,26,27,28,29], gesture synthesis is achieved by direct conversion from SWML or the SiGML notation into a VRML animation. GrieveSmith [30] uses the Stokoe notation for defining signs.…”

Section: Sign Description Approachesmentioning

confidence: 99%

Spanish Sign Language synthesis system

López-Colino

Colás

2012

Journal of Visual Languages & Computing

View full text Add to dashboard Cite

Esta es la versión de autor de la comunicación de congreso publicada en: This is an author produced version of a paper published in: AbstractThis work presents a new approach to the synthesis of Spanish Sign Language (LSE). Its main contributions are the use of a centralized relational database for storing sign descriptions, the proposal of a new input notation and a new avatar design, the skeleton structure of which improves the synthesis process.The relational database facilitates a highly detailed phonologic description of the signs that, include parameter synchronization and timing. The centralized database approach has been introduced to allow the representation of each sign to be validated by the LSE national institution, FCNSE.The input notation, designated HLSML, presents multiple levels of abstraction compared with current input notations. Redesigned input notation is used to simplify the description and the manual definition of LSE messages.Synthetic messages obtained using our approach have been evaluated by deaf users; in this evaluation a maximum recognition rate of 98.5% was obtained for isolated signs and a recognition rate of 95% was achieved for signed sentences.

show abstract

Linguistic modelling and language-processing technologies for Avatar-based sign language presentation

Cited by 114 publications

References 12 publications

Evaluation of a psycholinguistically motivated timing model for animations of american sign language

Evaluation of a psycholinguistically motivated timing model for animations of american sign language

An Open Web Platform for Rule-Based Speech-to-Sign Translation

Spanish Sign Language synthesis system

Contact Info

Product

Resources

About