An Audiovisual Talking Head for Augmented Speech Generation: Models and Animations Based on a Real Speaker’s Articulatory Data

Augmented Reality Talking Heads as a Support for Speech Perception and Production

“…Augmented reality displays of the face have been tested for both speech perception (Badin et al, 2008;Engwall, 2010;Engwall & Wik, 2009a;Grauwinkel et al, 2007;Kröger et al, 2008;Wik & Engwall, 2008) and speech production (Engwall, 2008;Engwall & Bälter, 2007;Engwall et al, 2006;Fagel & Madany, 2008;Massaro et al, 2008;Massaro & Light, 2003;. These studies have used different displays to visualize the intraoral articulation, as examplified in Fig.…”

“…1 and summarized in Table 1. The list excludes the epiglottis and the larynx, which are only shown in the studies by Badin et al (2008) and Kröger et al (2008).…”

“…To overcome such problems, Cohen et al (1998) proposed to supplement the face view with a separate display that shows the regions were the tongue is in contact with the palate. This additional display would however split the learner's visual attention and it has not been used in the subsequent studies by Massaro et al The alternative opted for by Badin et al (2008), Engwall (2010); Engwall & Wik (2009a); Wik & Engwall (2008) and Massaro et al (2008) is to concentrate on the midsagittal outline of the palate to facilitate the perception of the distance between the tongue and the palate. Engwall (2010); Engwall & Wik (2009a); Wik & Engwall (2008) simplified the display further by showing the tongue and jaw moving inside a dark oral cavity, with the limit of the transparent skin region corresponding to the palate outline ( Fig.…”

“…For the appearance, several researchers, e.g., Badin et al (2008); Massaro et al (2008) intentionally depart from realism by choosing contrasting colours for the different articulators. No user study has yet been performed to investigate whether viewers prefer easier discrimination between articulators or caricaturized realism.…”

“…(c) Viewer position inside the talking head, similar to one display in Massaro & Light (2003). (d) Front half of the head removed, similar to the display in Badin et al (2008). (e) Displaying the vocal tract only, similar to the display in Kröger et al (2008).…”