Eye movements track prioritized auditory features in selective attention to natural speech

Abstract

Over the last decades, cognitive neuroscience has identified a distributed set of brain regions that are critical for attention. Strong anatomical overlap with brain regions critical for oculomotor processes suggests a joint network for attention and eye movements. However, the role of this shared network in complex, naturalistic environments remains understudied. Here, we investigated eye movements in relation to (un)attended sentences of natural speech. Combining simultaneously recorded eye tracking and magnetoencephalographic data with temporal response functions, we show that gaze tracks attended speech, a phenomenon we termed ocular speech tracking. Ocular speech tracking even differentiates a target from a distractor in a multi-speaker context and is further related to intelligibility. Moreover, we provide evidence for its contribution to neural differences in speech processing, emphasizing the necessity to consider oculomotor activity in future research and in the interpretation of neural differences in auditory cognition.

Prediction tendency, eye movements, and attention in a unified framework of neural speech tracking

“…Based on the joint findings of the present as well as its preceding studies (Gehmacher et al, 2024, Schubert et al, 2023, we propose a unified working model in which anticipatory predictions as well as active sensing work (independently) together to support auditory speech perception (see Figure 6 for a schematic illustration). We suggest that anticipatory predictions about a feature help to interpret auditory information at different levels along the perceptual hierarchy.…”

“…To investigate the contribution of eye movements to neural speech tracking, we approached a mediation analysis similar to Gehmacher et al (2024). The TRFs that we obtained from these encoding models can be interpreted as time-resolved weights for a predictor variable that aims to explain a dependent variable (very similar to beta-coefficients in classic regression analyses).…”

“…If model weights are significantly reduced by the inclusion of true eye movements into the model in comparison to a model with a time-shuffled version of the same predictor (i.e. c' < c), this indicates that a meaningful part of the relationship between the speech envelope and neural responses was mediated by eye movements (for further details see also Gehmacher et al, 2024).…”

“…For all models (as in Gehmacher et al, 2024 andSchubert et al, 2023), we used the weaklyor non-informative default priors of Bambi (Capretto et al, 2022) and specified a more robust Student-T response distributions instead of the default gaussian distribution. To summarise model parameters, we report regression coefficients and the 94% high density intervals (HDI) of the posterior distribution (the default HDI in Bambi).…”

“…In another recent study from our lab (Gehmacher et al, 2024), we showed that eye movements continuously track intensity fluctuations of attended natural speech, a phenomenon we termed ocular speech tracking. We further established links to increased intelligibility with stronger ocular tracking and provided evidence that eye movements and neural activity share contributions to speech tracking.…”