Alignment to visual speech information

Miller, Rachel M.; Sanchez, Kauyumari; Rosenblum, Lawrence D.

doi:10.3758/app.72.6.1614

Cited by 39 publications

(41 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The results of R. M. Miller et al (2010) and Sanchez et al (2010) are consistent with evidence from the speech perception literature for the multimodal nature of speech perception (for a review, see Rosenblum, 2008). It has long been understood that visible speech information can change the perception of auditory speech (e.g., McGurk & MacDonald, 1976).…”

supporting

confidence: 78%

“…As we previously discussed, evidence already suggests that the information to which perceivers align is available in visual displays of speech (R. M. Miller et al, 2010;Sanchez et al, 2010). By introducing noise to the auditory modality, perceivers may be forced to rely more on the information available within the visual modality.…”

mentioning

confidence: 92%

“…Some have argued that phonetic convergence may serve as evidence for a link between the perception and production mechanisms of speech, which may be suggestive of a common currency between the processes of speech perception and speech production (for a review, see Fowler, 2004). The crossmodal nature of the findings of R. M. Miller et al (2010) and Sanchez et al (2010) suggest that the information to which perceivers converge (and the information that raters use to assess convergence) is available across sensory modalities, perhaps taking the form of amodal articulatory gestures. As such, this articulatory information would serve as the common currency between the processes of speech perception and speech production (Fowler, 2004;Shockley et al, 2004).…”

mentioning

confidence: 99%

“…The convergence literature has shown complicated interactions between model and shadower gender, as well as social-relationship factors. Where some studies have reported significantly greater phonetic convergence among female participants (e.g., R. M. Miller et al, 2010;Namy, Nygaard, & Sauerteig, 2002), others have reported greater phonetic convergence among male participants (e.g., Pardo, 2006;Pardo et al, 2014), and still others have reported no gender differences at all (e.g., Pardo et al, 2013). Even within each of the studies above, researchers have reported a great deal of variability in convergence among participants, depending on gender consistency between the model and the shadower, as well as the social purpose established between the interlocutors (e.g., R. M. Miller et al, 2010;Namy et al, 2002;Pardo et al, 2013).…”

mentioning

confidence: 99%

“…All of this research was conducted on normal-hearing participants with no formal lipreading experience. For example, R. M. Miller et al (2010) had perceivers shadow tokens that were presented both auditory-only and visual-only (showing an articulating face). Naïve perceptual judges rated the shadowed utterances of heard speech as sounding more like the auditory utterances of the model.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Visibility of speech articulation enhances auditory phonetic convergence

Dias

Rosenblum

2015

Atten Percept Psychophys

Self Cite

View full text Add to dashboard Cite

Talkers automatically imitate aspects of perceived speech, a phenomenon known as phonetic convergence. Talkers have previously been found to converge to auditory and visual speech information. Furthermore, talkers converge more to the speech of a conversational partner who is seen and heard, relative to one who is just heard (Dias & Rosenblum Perception, 40, 1457-1466, 2011. A question raised by this finding is what visual information facilitates the enhancement effect. In the following experiments, we investigated the possible contributions of visible speech articulation to visual enhancement of phonetic convergence within the noninteractive context of a shadowing task. In Experiment 1, we examined the influence of the visibility of a talker on phonetic convergence when shadowing auditory speech either in the clear or in low-level auditory noise. The results suggest that visual speech can compensate for convergence that is reduced by auditory noise masking. Experiment 2 further established the visibility of articulatory mouth movements as being important to the visual enhancement of phonetic convergence. Furthermore, the word frequency and phonological neighborhood density characteristics of the words shadowed were found to significantly predict phonetic convergence in both experiments. Consistent with previous findings (e.g., Goldinger Psychological Review, 105, 251-279, 1998), phonetic convergence was greater when shadowing low-frequency words. Convergence was also found to be greater for low-density words, contrasting with previous predictions of the effect of phonological neighborhood density on auditory phonetic convergence (e.g., Pardo, Jordan, Mallari, Scanlon, & Lewandowski Journal of Memory and Language, 69, 183-195, 2013). Implications of the results for a gestural account of phonetic convergence are discussed.

show abstract

supporting

confidence: 78%

mentioning

confidence: 92%

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Visibility of speech articulation enhances auditory phonetic convergence

Dias

Rosenblum

2015

Atten Percept Psychophys

Self Cite

View full text Add to dashboard Cite

show abstract

Primacy of Multimodal Speech Perception for the Brain and Science

Rosenblum

Dorsi

2021

The Handbook of Speech Perception

View full text Add to dashboard Cite

The listening talker: A review of human and algorithmic context-induced modifications of speech

Cooke

King

Garnier

et al. 2014

Computer Speech & Language

View full text Add to dashboard Cite

International audienceSpeech output technology is finding widespread application, including in scenarios where intelligibility might be compromised - at least for some listeners - by adverse conditions. Unlike most current algorithms, talkers continually adapt their speech patterns as a response to the immediate context of spoken communication, where the type of interlocutor and the environment are the dominant situational factors influencing speech production. Observations of talker behaviour can motivate the design of more robust speech output algorithms. Starting with a listener-oriented categorisation of possible goals for speech modification, this review article summarises the extensive set of behavioural findings related to human speech modification, identifies which factors appear to be beneficial, and goes on to examine previous computational attempts to improve intelligibility in noise. The review concludes by tabulating 46 speech modifications, many of which have yet to be perceptually or algorithmically evaluated. Consequently, the review provides a roadmap for future work in improving the robustness of speech output

show abstract

Alignment to visual speech information

Cited by 39 publications

References 48 publications

Visibility of speech articulation enhances auditory phonetic convergence

Visibility of speech articulation enhances auditory phonetic convergence

Primacy of Multimodal Speech Perception for the Brain and Science

The listening talker: A review of human and algorithmic context-induced modifications of speech

Contact Info

Product

Resources

About