Neurophysiological Evidence for Semantic Processing of Irrelevant Speech and Own-Name Detection in a Virtual Café

Brown, Adi; Pinto, Danna; Burgart, Ksenia; Zvilichovsky, Yair; Zion-Golumbic, Elana

doi:10.1523/jneurosci.1731-22.2023

Cited by 6 publications

(6 citation statements)

References 116 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Not only was performance worse and speech tracking reduced in the intermittent noise condition, but in this condition we also found an increase in skin conductance, characterized by more frequent phasic responses and a trend toward higher tonic responses, relative to the quiet and continuous noise conditions. These physiological responses reflect heightened activation of the sympathetic nervous system, which is known to respond to salient stimuli, both task-relevant and in the background (Brown et al, 2023; Dawson et al, 1989; Filion et al, 1991; Mueller-Pfeiffer et al, 2014), and is also engaged in conditions requiring high listening effort (Borghini & Hazan, 2018; Mackersie & Cones, 2011; et al, 2017).…”

Section: Discussionmentioning

confidence: 99%

“…The ability to move your eyes freely, is thought to be a central mechanism in controlling and focusing attention (Craighero & Rizzolatti, 2005). Past VR studies have shown that when given this freedom, some participants perform frequent spontaneous gaze-shifts around the environment, while others keep their eyes fixed on the target speaker (Brown et al, 2023; Shavit-Cohen & Zion Golumbic, 2019). When processing audiovisual speech in noise, past studies have shown people tend to focus their gaze more intently on speaker’s face, and particularly on the mouth area, ostensibly to utilize visual cues to overcome the acoustic degradation of the speech (Buchan et al, 2008; Król, 2018; Šabić et al, 2020).…”

Section: Discussionmentioning

confidence: 99%

“…In most studies, the speech stimuli consist of single syllables/words or simple sentences, tasks consist of word-by-word recognition, and the noises used are artificial and highly controlled in their spectro-temporal properties (Buss et al, 2003(Buss et al, , 2004Cooke, 2006;Fogerty et al, 2018;Howard-Jones & we used long segments of continuous speech presented in a realistic audiovisual context, we tested its overall comprehension rather than word-for-word recognition, and used naturally-recorded construction noise that is less controlled and perhaps more abrasive its acoustic properties than the modulated noises used in glimpsing studies (Fogerty et al, 2018). These design choices are part of our attempt to study speech processing in noise under more ecologically-relevant contexts (Brown et al, 2023;Deringer & Hanley, 2021;Parsons, 2015;Risko et al, 2016;Shavit-Cohen & Zion Golumbic, 2019). However, they also probably account for the discrepancies in findings relative to more traditional psychoacoustic experiments, and raise the possibility that while the glimpsing model may hold true for local perception of speech-elements, intermittent noise remains highly disruptive and incurs high cognitive costs when attempting to focus ones' attention on continuous speech for long periods of time.…”

Section: Why Is Intermittent Noise More Disruptive?mentioning

confidence: 99%

“…These do not fully capture the sensory and cognitive challenges of real-life environments, which are inherently multisensory, dynamic and highly contextual and therefore limited in their ecological validity (Holleman et al, 2020). In attempt to enhance our understanding of how the brain deals with speech in realistic noisy environments, and specifically during learning, here we use a Virtual Reality (VR) Classroom experimental design to simulates a realistic learning context and study the effects of different types of background noise on neural processing, physiological responses and comprehension of the lesson (Adams et al, 2009; Brown et al, 2023; Liu et al, 2021; Tromp et al, 2018).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

An ecological investigation of the effect of background noise on speech processing in a Virtual Classroom

Levy,

Korisky,

Zvilichovsky

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

Many real-life situations can be extremely noisy. Psychoacoustic studies have shown that background noise can have a detrimental effect on the ability to process and understand speech. However, most studies use stimuli and task designs that are highly artificial, limiting their generalization to more realistic contexts. Moreover, to date, we do not fully understand the neurophysiological consequences of trying to pay attention to speech in a noisy place.To address this lab to real-life gap and increase the ecological validity of speech in noise research, here we introduce a novel audiovisual Virtual Reality (VR) experimental platform. Combined with neurophysiological measurements of neural activity (EEG), eye-gaze and skin conductance (GSR) we studied the effects of background noise in a realistic context where the ability to process and understand continuous speech is especially important: A VR Classroom.Participants (n=32) sat in a VR Classroom and were told to pay attention to mini-lecture segments by a virtual teacher. Trials were either Quiet or contained background construction noise, emitted from outside the classroom window, which was either Continuous (drilling) or Intermittent (air hammers).Result show that background noise had a detrimental effect on learning outcomes, which was also accompanied by reduced neural tracking of the teacher’s speech. Comparison of the two noise types showed that the intermittent construction noise was more disruptive than continuous noise, as index by both behavioral and neural measures, and it also elicited higher skin-conductance levels, reflecting heightened arousal. Interesting, eye-gaze dynamics were not affected by the presence of noise.This study advances our understanding of the neurophysiological effects of background noise and extends it to more ecologically relevant contexts. It also emphasizes the role that temporal dynamics play for processing speech in noise, highlighting the need to consider the features of realistic noises, as we expand speech in noise research to increasingly realistic circumstances.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Why Is Intermittent Noise More Disruptive?mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An ecological investigation of the effect of background noise on speech processing in a Virtual Classroom

Levy,

Korisky,

Zvilichovsky

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Using a linear approach for speech-tracking analysis and auditory attention decoding (Crosse et al, 2016; Zion Golumbic, Ding, et al, 2013), we investigated how the neural representation of the concurrent speech stimuli was modulated by their status as target/non-target, and whether this was affected once they switched roles. Moreover, our choice to use natural audiovisual speech and to present audio in a spatially-realistic free-field manner, increases the ecological validity of this study, relative to past studies that mostly use audio-only speech presented through headphones (Brown et al, 2023; Freyman et al, 2001; Keidser et al, 2020; Ross et al, 2007; Shavit-Cohen & Zion Golumbic, 2019; Tye-Mmurray et al, 2016; Uhrig et al, 2022).…”

Section: Introductionmentioning

confidence: 99%

Neural Speech-Tracking During Selective Attention and Attention-Switching Between Concurrent Talkers: a spatially realistic audiovisual study

Har-shai Yahav,

Korisky,

Vaknin Harel

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

Selective attention to a talker in multi-talker scenarios, is associated with enhanced neural speech-tracking of the target talker, relative to competing non-target talkers. This response is proposed to reflect the prioritization and preferential processing of the target speech in auditory and language related regions. Here we investigated whether this neural-bias for tracking target speech, persists also after listeners actively switch their attention to another target talker. Using a spatially-realistic audiovisual design, we studied responses at both the group and individual level.Participants watched a video lecture (target speech) on a screen in front of them, which they were instructed to pay attention to and answered comprehension questions about its content. Concurrently, audio from an additional lecture, by a different talker, was played though a loudspeaker to their left, which they were told to ignore (non-target speech). Importantly, in the middle of the experiment, the lectures serving as target and non-target switched roles: the lecture that was formerly ‘non-target’ was presented as a video on the central screen and became the ‘target’, and vice versa. We compared behavioral performance and neural speech-tracking of both lectures before and after the switch, to test how effectively participants switched their attention and whether there are carry-over effects from before the switch.At the group level, we found no significant differences in performance or in the neural-bias towards the target talker before vs. after the switch. This implies that, generally speaking, selective attention was just as effective after the switch as it was before. However, analysis of individual level data revealed a more complex and diverse pattern. We found that the neural group level results were driven by only 25% of participants, who showed consistent neural-bias towards the target talker both before and after the switch. However, 50% of participants showed neural-bias towards the target talker only in one half of the experiment, and an additional 25% showed no difference in the neural representation of the target vs. non-target speech, before or after the switch. We further show that these individual differences cannot be ‘trivially’ explained by poor signal quality, nor are they associated with behavioral performance. Rather, they suggest that neural-bias towards the target talker is not a prerequisite for selective attention performance, and in many cases target and non-target speech are co-represented in the neural signal. Whether this variability reflects the employment of different listening strategies or insufficient sensitivity of the neural-bias speech-tracking metric remains to be explored in future studies. However, these results challenge the assumptions that modulation of the speech-tracking response can be used as a one-to-one proxy for selective attention to speech in multi-talker contexts.

show abstract

Linguistic Processing of Unattended Speech Under a Cocktail Party Listening Scenario

Lu,

Wu,

Zhang

et al. 2024

J Speech Lang Hear Res

View full text Add to dashboard Cite

Purpose: In the context of a cocktail party listening environment, the processing of different linguistic hierarchy levels in unattended speech and their influence on target speech recognition remain controversial. This study aims to investigate how different levels of linguistic structures (such as syllable, word, and sentence) in competing speech influence the recognition of target speech in a speech-to-speech masking situation. Method: Thirty-six participants were instructed to recognize target speech when it was masked by competing speech varied in masking types across syllables, words, and sentence. The perceived spatial location was altered to examine the interaction between linguistic unmasking effects and spatial unmasking effects. Recognition performance (i.e., intelligibility threshold) was determined by fitting psychometric functions to the recognition accuracies across four signal-to-noise ratios (−14, −10, −6, and − 2 dB) to evaluate each subject's ability to cope with challenging listening conditions. Results: We revealed a significant decline in target speech recognition when the masking speech was linguistically structured and intelligible. Specifically, masking speech with higher linguistic complexity, such as coherent sentences, resulted in more significant interference compared to those with lower complexity, like sequences of syllables. The linguistic release from masking, resulting from a decrease in linguistic complexity of maskers shifting from sentences to syllables, was found to be correlated with, and also linearly additive to, the spatial release from masking due to the spatial separation of the masker and target. Conclusions: These findings illustrate the influence of linguistic complexity in masking speech on the recognition of target speech, suggesting the involvement of higher-level linguistic processing of irrelevant speech in noisy environment.

show abstract

Neurophysiological Evidence for Semantic Processing of Irrelevant Speech and Own-Name Detection in a Virtual Café

Cited by 6 publications

References 116 publications

An ecological investigation of the effect of background noise on speech processing in a Virtual Classroom

An ecological investigation of the effect of background noise on speech processing in a Virtual Classroom

Neural Speech-Tracking During Selective Attention and Attention-Switching Between Concurrent Talkers: a spatially realistic audiovisual study

Linguistic Processing of Unattended Speech Under a Cocktail Party Listening Scenario

Contact Info

Product

Resources

About