Speech face perception is locked to anticipation in speech production

Troille, Emilie; Cathiard, Marie‐Agnès; Abry, Christian

doi:10.1016/j.specom.2009.12.005

Cited by 10 publications

(9 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Lip area values, as calculated on the still image of the lips at vowel midpoint, were 1.4 and 1.3 cm 2 , respectively, for both vowels of /izi/, and lip area values were 1.2 and 0.4 cm 2 , respectively, for both vowels of /izy/. Those geometrical values are in line with those reported for French speakers [9,14]. The fricative consonant /z/ was chosen because it is phonologically a nonlabial consonant, thus allowing the anticipatory rounded gesture associated with the phonologically labial /y/ vowel to be produced through this consonant.…”

Section: Methodssupporting

confidence: 89%

“…When presented with truncated stimuli (gating paradigm) in the visual-only condition, participants can identify the rounded vowel when the lip area decreased to approximately 1 cm 2 , before the change in formant structure associated with lip rounding [8,9]. Thus, the varying shapes of the speaker's lips provide the listener with anticipatory cues that are functional in the speech perception process.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Effects of Congenital Visual Deprivation on the Auditory Perception of Anticipatory Labial Coarticulation

Ménard

Cathiard

Troille³

et al. 2015

Folia Phoniatr Logop

Self Cite

View full text Add to dashboard Cite

Objective: It has been shown previously that congenitally blind francophone adults had higher auditory discrimination scores than sighted adults. It is unclear, however, if, compared to their sighted peers, blind speakers display an increased ability to detect anticipatory acoustic cues. In this paper, this ability is investigated in both speaker groups. Methods: Using the gating paradigm, /izi/ and /izy/ sequences were truncated to include a variable duration of the vowel. The sequences were used as stimuli in an auditory identification test. Seventeen congenitally blind adults (9 females and 8 males) and 17 sighted controls were recruited. Their task was to identify the second vowel of the sequence. Results: Results show that all participants could reliably identify the rounded vowel prior to its acoustic onset, but steeper identification slopes were found for sighted listeners than for blind listeners. Conclusion: The difference in identification slopes likely suggests that sighted speakers display finer abilities to perceptually follow the decreasing values of the frication noise, compared to blind speakers.

show abstract

Section: Methodssupporting

confidence: 89%

Section: Introductionmentioning

confidence: 99%

Effects of Congenital Visual Deprivation on the Auditory Perception of Anticipatory Labial Coarticulation

Ménard

Cathiard

Troille³

et al. 2015

Folia Phoniatr Logop

Self Cite

View full text Add to dashboard Cite

show abstract

“…But the next question is to know if the auditory and visual systems are able to process the information efficiently as soon as it is available. This is actually not always the case, and in gating experiments on the visual vs. auditory identification of coarticulated sequences, Troille et al [37] display in some configurations a lead of audition on vision which can reach up to 40 ms, because of the poor visibility of some articulatory gestures. This leads the authors to claim that they have discovered a case where “speech can be heard before it is seen”.…”

Section: Discussionmentioning

confidence: 99%

“…In summary, there are actually a variety of situations from audio lead (estimated to 40 ms in [37]) to visual lead (which can reach more than 200 ms). In their study of mutual information between audio and video parameters on speech sequences, Feldhoffer et al [38] show that mutual information is maximal for some audio and video parameters when it incorporates a video lead up to 100 ms.…”

Section: Discussionmentioning

confidence: 99%

No, There Is No 150 ms Lead of Visual Speech on Auditory Speech, but a Range of Audiovisual Asynchronies Varying from Small Audio Lead to Large Audio Lag

Schwartz

Savariaux

2014

PLoS Comput Biol

View full text Add to dashboard Cite

An increasing number of neuroscience papers capitalize on the assumption published in this journal that visual speech would be typically 150 ms ahead of auditory speech. It happens that the estimation of audiovisual asynchrony in the reference paper is valid only in very specific cases, for isolated consonant-vowel syllables or at the beginning of a speech utterance, in what we call “preparatory gestures”. However, when syllables are chained in sequences, as they are typically in most parts of a natural speech utterance, asynchrony should be defined in a different way. This is what we call “comodulatory gestures” providing auditory and visual events more or less in synchrony. We provide audiovisual data on sequences of plosive-vowel syllables (pa, ta, ka, ba, da, ga, ma, na) showing that audiovisual synchrony is actually rather precise, varying between 20 ms audio lead and 70 ms audio lag. We show how more complex speech material should result in a range typically varying between 40 ms audio lead and 200 ms audio lag, and we discuss how this natural coordination is reflected in the so-called temporal integration window for audiovisual speech perception. Finally we present a toy model of auditory and audiovisual predictive coding, showing that visual lead is actually not necessary for visual prediction.

show abstract

“…Consequently, certain specific aspects of our data may not generalize to other speech sounds, tokens, speakers, etc. These factors have been shown to influence the outcome of, e.g., gating studies (Troille, Cathiard, & Abry, 2010). However, the main findings of the current study seem likely to hold across additional contexts.…”

Section: Discussionmentioning

confidence: 99%

Timing in audiovisual speech perception: A mini review and new psychophysical data

Venezia

Thurman

Matchin

et al. 2015

Atten Percept Psychophys

View full text Add to dashboard Cite

Recent influential models of audiovisual speech perception suggest that visual speech aids perception by generating predictions about the identity of upcoming speech sounds. These models place stock in the assumption that visual speech leads auditory speech in time. However, it is unclear whether and to what extent temporally-leading visual speech information contributes to perception. Previous studies exploring audiovisual-speech timing have relied upon psychophysical procedures that require artificial manipulation of cross-modal alignment or stimulus duration. We introduce a classification procedure that tracks perceptually-relevant visual speech information in time without requiring such manipulations. Participants were shown videos of a McGurk syllable (auditory /apa/ + visual /aka/ = perceptual /ata/) and asked to perform phoneme identification (/apa/ yes-no). The mouth region of the visual stimulus was overlaid with a dynamic transparency mask that obscured visual speech in some frames but not others randomly across trials. Variability in participants' responses (∼35% identification of /apa/ compared to ∼5% in the absence of the masker) served as the basis for classification analysis. The outcome was a high resolution spatiotemporal map of perceptually-relevant visual features. We produced these maps for McGurk stimuli at different audiovisual temporal offsets (natural timing, 50-ms visual lead, and 100-ms visual lead). Briefly, temporally-leading (∼130 ms) visual information did influence auditory perception. Moreover, several visual features influenced perception of a single speech sound, with the relative influence of each feature depending on both its temporal relation to the auditory signal and its informational content.

show abstract

Speech face perception is locked to anticipation in speech production

Cited by 10 publications

References 13 publications

Effects of Congenital Visual Deprivation on the Auditory Perception of Anticipatory Labial Coarticulation

Effects of Congenital Visual Deprivation on the Auditory Perception of Anticipatory Labial Coarticulation

No, There Is No 150 ms Lead of Visual Speech on Auditory Speech, but a Range of Audiovisual Asynchronies Varying from Small Audio Lead to Large Audio Lag

Timing in audiovisual speech perception: A mini review and new psychophysical data

Contact Info

Product

Resources

About