What Comes After /f/? Prediction in Speech Derives From Data-Explanatory Processes

Abstract

Speech perception is a difficult problem most listeners solve effortlessly. Acoustic cues are short-lived and highly variable. Here we demonstrate that part of the solution is predictions modulated by high-level expectations. Listeners heard isolated fricatives (e.g., “s”, “sh”) and predicted the upcoming vowel. Accuracy was above chance, suggesting fine-grained detail in the signal can be used for prediction. A second group performed the same task but saw a still face and a letter corresponding to the fricative. This group performed markedly better suggesting that high-level knowledge modulates prediction by helping listeners form expectations about what the fricative should have sounded like. This suggests a form of data explanation operating in speech perception: as listeners account for variance due to their knowledge of the talker and current phoneme, they use what’s left over to make more accurate predictions about the next sound.

Evaluating the sources and functions of gradiency in phoneme categorization: An individual differences approach.

“…For example, a stimulus with a VOT of 20 ms could be a /b/ in slow speech or a /p/ in fast speech. Despite over 40 years of research, speech scientists have identified few (if any) acoustic cues that unambiguously identify a phoneme across different contexts (e.g., McMurray & Jongman, 2015; Ohala, 1996). …”

“…Such processes may underlie the well-known trading relations that have been documented in speech perception (Repp, 1982; Summerfield & Haggard, 1977; Winn, Chatterjee, & Idsardi, 2013). This kind of combinatory process would also be necessary for accurately compensating for higher level contextual expectations—for example, recoding pitch relative to the talker’s mean pitch (McMurray & Jongman, 2011, 2015). …”

Prediction in aging language processing

“…Hierarchical predictive frameworks have been recently extended to explain phenomena in language processing (Farmer et al, 2013;Kuperberg & Jaeger, 2016;McMurray & Jongman, 2016). According to these frameworks, expectations at higher levels of processing (e.g., syntactic) create hypotheses and send those to lower levels of representation (e.g., word recognition).…”

Individual differences in speech perception

“…First, processes like coarticulation and assimilation leave fine-grained, subcategorical traces in the signal (e.g., Gow, 2001), which can be used to anticipate upcoming speech sounds speeding up processing. Multiple studies suggest that listeners take advantage of anticipatory coarticulatory information in this way (Gow, 2001;Mahr, McMillan, Saffran, Ellis Weismer, & Edwards, 2015;McMurray & Jongman, 2015;Salverda, Kleinschmidt, & Tanenhaus, 2014;Yeni-Komshian, 1981). However, as these modifications are largely within-category, such anticipation is only possible if listeners are sensitive to this fine-grained detail.…”

“…For example, a stimulus with a voice onset time (VOT) of 20 ms could be a /b/ in slow speech or a /p/ in fast speech. In fact, despite over 40 years of research, phoneticians and speech scientists have identified few (if any) acoustic cues that unambiguously identify a phoneme across different contexts (e.g., McMurray & Jongman, 2015;Ohala, 1996).…”