Listening with Great Expectations: An Investigation of Word Form Anticipations in Naturalistic Speech

Abstract

The event-related potential (ERP) component named phonological mismatch negativity (PMN) arises when listeners hear an unexpected word form in a spoken sentence [1]. The PMN is thought to reflect the mismatch between expected and perceived auditory speech input. In this paper, we use the PMN to test a central premise in the predictive coding framework [2], namely that the mismatch between prior expectations and sensory input is an important mechanism of perception. We test this with natural speech materials containing approximately 50,000 word tokens. The corresponding EEG-signal was recorded while participants (n = 48) listened to these materials. Following [3], we quantify the mismatch with two word probability distributions (WPD): a WPD based on preceding context, and a WPD that is additionally updated based on the incoming audio of the current word. We use the between-WPD cross entropy for each word in the utterances and show that a higher cross entropy correlates with a more negative PMN. Our results show that listeners anticipate auditory input while processing each word in naturalistic speech. Moreover, complementing previous research, we show that predictive language processing occurs across the whole probability spectrum.

Speech register influences listeners’ word expectations

“…Yan et al (2017) discuss both Delong et al (2005) and a preprint version of Nieuwland's study and argue that the findings from this replication study can also be interpreted as in line with a prediction account; for example, the correlation between cloze values and the N400 was replicated. We interpret the combined literature (see also Wicha et al, 2004;Van Berkum et al, 2005) as supportive of anticipatory processing, especially because new evidence supporting phonological pre-activation can be found in Bentum et al (2019b); see also Poulton & Nieuwland (2022) for a critical view).…”

Word Competition: An Entropy-Based Approach in the DIANA Model of Human Word Comprehension

“…In this paper we present results showing that entropy, next to the conventional control predictors, is a useful measure in two ways: (a) as predictor in a data-oriented lmer() model, and (b) as theoretical underpinning of DIANA's choiceRT rule, and thereby closely related to the way DIANA processes the speech signal and produces dynamically changing lists of word (cohort) hypotheses. There is a close connection with the cross entropy between the two probability density functions pdf0 and pdfT which unfold over time ( [19]).…”

“…DIANA (e.g., [14]) has been used to simulate RTs in the Dutch BALDEY database [15] and the large-scale North American English data (MALD) [16], for compounds [17], to differentiate predictors in regression models [18], and for modelling the cross-entropy as a significant predictor for modeling ERP components during comprehension of continuous speech in a recent EEG experiment [19].…”

“…That there is such a relation is suggested by other research. [19]'s ERP study using over 2 million EEG traces has shown that DIANA's activation of word cohorts for gated stimuli audio[0 : T ] for varying T is closely related to the cross entropy between two probability density functions pdf0 and pdfT .…”

DIANA, a Process-Oriented Model of Human Auditory Word Recognition

“…Neurophysiological research using the phonetic mismatch negativity (a measure of mismatch between expected and actual phonetic input) in EEG traces has shown that, from word onset onwards, listeners develop expectations about which word is uttered, based on both the bottom-up information from the acoustic signal, and the top-down expectations from the (linguistic) context [51,52]. In DIANA, the words' activations are also based on a combination of both types of evidence.…”