The Timing and Effort of Lexical Access in Natural and Degraded Speech

Abstract

Understanding speech is effortless in ideal situations, and although adverse conditions, such as caused by hearing impairment, often render it an effortful task, they do not necessarily suspend speech comprehension. A prime example of this is speech perception by cochlear implant users, whose hearing prostheses transmit speech as a significantly degraded signal. It is yet unknown how mechanisms of speech processing deal with such degraded signals, and whether they are affected by effortful processing of speech. This paper compares the automatic process of lexical competition between natural and degraded speech, and combines gaze fixations, which capture the course of lexical disambiguation, with pupillometry, which quantifies the mental effort involved in processing speech. Listeners’ ocular responses were recorded during disambiguation of lexical embeddings with matching and mismatching durational cues. Durational cues were selected due to their substantial role in listeners’ quick limitation of the number of lexical candidates for lexical access in natural speech. Results showed that lexical competition increased mental effort in processing natural stimuli in particular in presence of mismatching cues. Signal degradation reduced listeners’ ability to quickly integrate durational cues in lexical selection, and delayed and prolonged lexical competition. The effort of processing degraded speech was increased overall, and because it had its sources at the pre-lexical level this effect can be attributed to listening to degraded speech rather than to lexical disambiguation. In sum, the course of lexical competition was largely comparable for natural and degraded speech, but showed crucial shifts in timing, and different sources of increased mental effort. We argue that well-timed progress of information from sensory to pre-lexical and lexical stages of processing, which is the result of perceptual adaptation during speech development, is the reason why in ideal situations speech is perceived as an undemanding task. Degradation of the signal or the receiver channel can quickly bring this well-adjusted timing out of balance and lead to increase in mental effort. Incomplete and effortful processing at the early pre-lexical stages has its consequences on lexical processing as it adds uncertainty to the forming and revising of lexical hypotheses.

Hearing aid experience and background noise affect the robust relationship between working memory and speech recognition in noise

“…These findings are in line with R€ onnberg et al 2013, which states that based on the ELU model, additional WMC is important for listeners to understand hearing aid processed speech when matching supposedly is disrupted by low fidelity of the signal. Wagner, Toffanin, and Başkent (2016) studied the impact of degraded speech on lexical access, and they found that lexical processing was more time consuming and effortful when speech was degraded than natural speech. In other words, the deployment of cognitive resources in speech understanding increases as the fidelity of the input signal decreases.…”

Measuring the contribution to cognitive load of each predicted vocoder speech parameter in DNN-based speech synthesis

“…Subsequently, the Event Related Pupil Dilation (ERPD) percentage was computed. This was calculated using the equation in [16]. Some problems with the eye-tracker were experienced during data collection where a warning of the pupil size was shown.…”

Multitasking with typical use of hearing aid noise reduction in older listeners

“…multitasking). The rationale for including secondary task responses in the silent period following the sentence is that the time course of attention and lexical access extends beyond stimulus presentation (Kahneman, 1973;Wagner et al, 2016).…”