Knowledge-based and signal-based cues are weighted flexibly during spoken language comprehension.

Abstract

During spoken language comprehension, listeners make use of both knowledge-based and signal-based sources of information, but little is known about how cues from these distinct levels of representational hierarchy are weighted and integrated online. In an eye-tracking experiment using the visual world paradigm, we investigated the flexible weighting and integration of morphosyntactic gender marking (a knowledge-based cue) and contextual speech rate (a signal-based cue). We observed that participants used the morphosyntactic cue immediately to make predictions about upcoming referents, even in the presence of uncertainty about the cue’s reliability. Moreover, we found speech rate normalization effects in participants’ gaze patterns even in the presence of preceding morphosyntactic information. These results demonstrate that cues are weighted and integrated flexibly online, rather than adhering to a strict hierarchy. We further found rate normalization effects in the looking behavior of participants who showed a strong behavioral preference for the morphosyntactic gender cue. This indicates that rate normalization effects are robust and potentially automatic. We discuss these results in light of theories of cue integration and the two-stage model of acoustic context effects.

A Compositional Neural Architecture for Language

“…The chief prediction regarding structure and meaning from the architecture is that low-frequency power and phase synchronization should increase as structure and meaning build up in time. This has been attested in the literature (Brennan & Martin, 2019;Kaufeld, Naumann, et al, 2019;Kaufeld, Ravenschlag, et al, 2019;Meyer, 2018;Ding et al, 2016;Meyer, Henry, Gaston, Schmuck, & Friederici, 2016;Bastiaansen et al, 2005Bastiaansen et al, , 2008 but needs more careful investigation. It is likely that low-frequency phase organization reflects the increasingly distributed nature of the neural assemblies being (de)synchronized as structure and meaning are inferred, rather than reflecting a phrasal or sentential oscillator.…”

“…Perceptual inference asserts that sensory cues activate latent representations in the neural system that have been learned through experience. 4 In line with this idea, there is everaccumulating evidence that "lower level" cues like speech rate and phoneme perception (e.g., Kaufeld, Ravenschlag, Meyer, Martin, & Bosker, 2019;Kaufeld, Naumann, Meyer, Bosker, & Martin, 2019;Heffner, Dilley, McAuley, & Pitt, 2013;Dilley & Pitt, 2010), morphology (e.g., Gwilliams, Linzen, Poeppel, & Marantz, 2018;Martin, Monahan, & Samuel, 2017), foveal and parafoveally processed orthography (e.g., Cutter, Martin, & Sturt, 2019;Veldre & Andrews, 2018;Schotter, Angele, & Rayner, 2012), as well as "higher level" sentential (e.g., Kutas, Ferreira, & Martin, 2018;Martin & McElree, 2008van Alphen & Table 1.…”

Temporal contrast effects in human speech perception are immune to selective attention

“…That is, in a slow context, syllables can disappear from perception 9,10 . This acoustic context effect induced by the surrounding speech rate, known as a temporal contrast effect or rate normalization, has been shown to influence a wide range of different duration-based phonological cues such as voice onset time (VOT; 11,12 ), formant transition duration 13 , vowel duration 14,15 , lexical stress 16 , and word segmentation 17,18 . In fact, a similar contrastive effect is found in the spectral domain: a sentence with a relatively low first formant (F1) can bias the perception of a following target with an ambiguous F1 (e.g., ambiguous between "bit" and "bet") towards a high F1 percept ("bet"; known as a spectral contrast effect or spectral normalization 6,19,20 ).…”

Listeners normalize speech for contextual speech rate even without an explicit recognition task

“…As such, speech rate modulated perception of VOT, whereas vowel cues, which followed the VOT contrast, were used later. Recently, evidence for the automaticity of rate normalization was found in a third eye-tracking study (Kaufeld et al, 2019). Kaufeld et al compared effects of knowledge-based (morphosyntactic gender marking) and signal-based (speech rate) cues in a two-alternative forced choice (2AFC) task, while also measuring participants' eye movements.…”