Intelligent animated agents for interactive language training

Abstract

This report describes a three-year project, now eight months old, to develop interactive learning tools for language training with profoundly deaf children. The tools combine four key technologies: speech recognition, developed at the Oregon Graduate Institute; speech synthesis, developed at the University of Edinburgh and modified at OGI; facial animation, developed at University of California, Santa Cruz; and face tracking and speech reading, developed at Carnegie Mellon University. These technologies are being combined to create an intelligent conversational agent; a three-dimensional face that produces and understands auditory and visual speech. The agent has been incorporated into CSLU Toolkit, a software environment for developing and researching spoken language systems. We describe our experiences in bringing interactive learning tools to classrooms at the Tucker-Maxon Oral School in Portland, Oregon, and the technological advances that are required for this project to succeed.

Improving students’ pronunciation through accent reduction software

“…Therefore, almost all software packages to teach pronunciation focus on accuracy at the expense of fluency and extended discourse. Seferoğlu (2003, p. 5) claims that “one of the main limitations of many of the computer assisted pronunciation software packages is that they are limited to presenting and practicing of segmental aspects (ie, individual sounds) of the language rather than suprasegmental aspects and connected speech.” Moreover, many commercially available software packages have been criticised because they were not designed based on sound pedagogical guidelines (Pennington, 1999; Warschauer & Healey, 1998) or they failed to adopt a multidisciplinary approach that includes teachers, linguists, speech experts, etc in the design process (Cole et al , 1998; Price, 1998).…”

Trainable Articulatory Control Models for Visual Speech Synthesis

“…The applications for this technology include virtual language tutors (Cole et al, 1998), communication aids for hard-of-hearing people (Agelfors et al, 1998), embodied conversational agents in spoken dialogue systems (Gustafson et al 2000) and talking computer game characters, to name only a few. Proper visual speech movements are often a crucial factor for the realism of such avatars, and in some applications, realistic speech movement constitute the main motivation for the technology (Agelfors et al, 1998;Siciliano et al, 2003).…”

The Pedagogy-Technology Interface in Computer Assisted Pronunciation Training

“…This may not necessarily be due to a lack of willingness, on the part of the developers, to include pedagogical guidelines in the design. It may simply be due to a failure to adopt a multidisciplinary approach involving speech technologists, linguists and language teachers (Cole et al 1998, Price 1998, or more fundamentally, to the absence of clear pedagogical guidelines that suit these types of environments.…”