Visual perception of anticipatory rounding gestures

Abstract

The mechanisms of anticipation for the rounding gesture have been repeatedly investigated in previous works (see, e.g., the controversy between Lubker and Gay concerning the extent of anticipation in Swedish versus American English, or the support found in the French language for Henke's “look ahead” model as exemplified by data from Benguerel's famous “sinistre structure”). Concerning the visual perception of such an anticipation, McGurk [in The Cognitive Representation of Speech, edited by T. Myers et al. (North-Holland, Amsterdam, 1981), p. 336] has briefly mentioned an experiment using reaction times in CV identification. He demonstrated that listeners do take visual information about anticipation into account, and identify CV syllables on the basis of lip movement information prior to their being perceived auditorily. In the present experiment, this same result is found for French, with a different experimental protocol, taking into account simultaneous acoustic and articulatory measurements. Here, /zV1zV2/ trajectories (V1, V2, = /i/ or /y/), were used and compared with auditory identification data obtained from gated signals with results of visual identification for front face video images taken every 20 ms along the V1 → V2 trajectory. The following results were found: (1) Lip area transitions clearly show the asymmetry of vowel-to-vowel gestures. The transition from /y/ to /i/ begins at the acoustic: onset of the consonant, while the transition from /i/ to /y/ can begin very early in the /i/; (2) this anticipation of the rounding gesture is clearly perceived visually by the subjects who are able to identify the /y/ vowel before the end of the /i/; and (3) visual detection of the rounding gesture thus comes prior to its auditory detection, which seems, in fact, disturbed by the acoustic mixture of the vocalic gesture (/i/ → /y/ or /y/ → /i/) and the consonantal gesture (/z/). Implications for the timing and perception of the vowel-to-vowel gesture are discussed.

Timing in audiovisual speech perception: A mini review and new psychophysical data

“…Cathiard, Lallouache, & Abry, 1996). The same visible gesture was perceived 40-60 ms ahead of the acoustic change when vowels were separated by a consonant (i.e., in a CVCV sequence; Escudier, Benoît, & Lallouache, 1990), and, furthermore, visual perception could be linked to articulatory parameters of the lips (Abry, Lallouache, & Cathiard, 1996). Additionally, accurate visual perception of bilabial and labiodental consonants in CV segments was demonstrated up to 80 ms prior to the consonant release (Smeele, 1994).…”