Neural Speech-Tracking During Selective Attention and Attention-Switching Between Concurrent Talkers: a spatially realistic audiovisual study

Abstract

Selective attention to a talker in multi-talker scenarios, is associated with enhanced neural speech-tracking of the target talker, relative to competing non-target talkers. This response is proposed to reflect the prioritization and preferential processing of the target speech in auditory and language related regions. Here we investigated whether this neural-bias for tracking target speech, persists also after listeners actively switch their attention to another target talker. Using a spatially-realistic audiovisual design, we studied responses at both the group and individual level.Participants watched a video lecture (target speech) on a screen in front of them, which they were instructed to pay attention to and answered comprehension questions about its content. Concurrently, audio from an additional lecture, by a different talker, was played though a loudspeaker to their left, which they were told to ignore (non-target speech). Importantly, in the middle of the experiment, the lectures serving as target and non-target switched roles: the lecture that was formerly ‘non-target’ was presented as a video on the central screen and became the ‘target’, and vice versa. We compared behavioral performance and neural speech-tracking of both lectures before and after the switch, to test how effectively participants switched their attention and whether there are carry-over effects from before the switch.At the group level, we found no significant differences in performance or in the neural-bias towards the target talker before vs. after the switch. This implies that, generally speaking, selective attention was just as effective after the switch as it was before. However, analysis of individual level data revealed a more complex and diverse pattern. We found that the neural group level results were driven by only 25% of participants, who showed consistent neural-bias towards the target talker both before and after the switch. However, 50% of participants showed neural-bias towards the target talker only in one half of the experiment, and an additional 25% showed no difference in the neural representation of the target vs. non-target speech, before or after the switch. We further show that these individual differences cannot be ‘trivially’ explained by poor signal quality, nor are they associated with behavioral performance. Rather, they suggest that neural-bias towards the target talker is not a prerequisite for selective attention performance, and in many cases target and non-target speech are co-represented in the neural signal. Whether this variability reflects the employment of different listening strategies or insufficient sensitivity of the neural-bias speech-tracking metric remains to be explored in future studies. However, these results challenge the assumptions that modulation of the speech-tracking response can be used as a one-to-one proxy for selective attention to speech in multi-talker contexts.