Transformation from auditory to linguistic representations across auditory cortex is rapid and attention dependent for continuous speech

Brodbeck, Christian; Le, Hong; Simon, Jonathan Z.

doi:10.1101/326785

Cited by 8 publications

(7 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, a later filter on the ignored signal must actively suppress distracting inputs. We found such a neural filter mechanism (Fig 4C&G) active in a time range which was previously attributed to processing of phonological (Di Liberto et al 2016, Brodbeck at al. 2018) as well as semantic features (Broderick at al.…”

Section: Discussionsupporting

confidence: 82%

Late cortical tracking of ignored speech facilitates neural selectivity in acoustically challenging conditions

Fiedler

Woestmann

Herbst

et al. 2017

Preprint

View full text Add to dashboard Cite

Listening requires selective neural processing of the incoming sound mixture, which in humans is borne out by a surprisingly clean representation of attended-only speech in auditory cortex. How this neural selectivity is achieved even at negative signal-to-noise ratios (SNR) remains unclear. We show that, under such conditions, a late cortical representation (i.e., neural tracking) of the ignored acoustic signal is key to successful separation of attended and distracting talkers (i.e., neural selectivity). We recorded and modelled the electroencephalographic response of 18 participants who attended to one of two simultaneously presented stories, while the SNR between the two talkers varied dynamically. The neural tracking showed an increasing early-to-late attention-biased selectivity. Importantly, acoustically dominant ignored talkers were tracked neurally by late involvement of fronto-parietal regions, which contributed to enhanced neural selectivity. This neural selectivity by way of representing the ignored talker poses a mechanistic neural account of attention under real-life acoustic conditions.

show abstract

Section: Discussionsupporting

confidence: 82%

Late cortical tracking of ignored speech facilitates neural selectivity in acoustically challenging conditions

Fiedler

Woestmann

Herbst

et al. 2017

Preprint

View full text Add to dashboard Cite

show abstract

“…Finally, the effects of melodic expectations exhibited different temporal and spatial dynamics from those that were measured in a similar manner for phonotactic-and semantic-level predictions in the case of natural speech perception (Brodbeck, Hong, et al 2018a;Broderick et al 2018;Di Liberto et al 2019). One noticeable difference is that the strongest TRF components for phonotactic-and semantic-TRFs (Broderick et al 2018 andDi Liberto et al 2019 respectively), both in centro-parietal scalp areas, were absent in the responses to melodic surprise and entropy.…”

Section: Discussionmentioning

confidence: 83%

Cortical encoding of melodic expectations in human temporal cortex

Liberto¹,

Pelofi²,

Bianco³

et al. 2019

Preprint

View full text Add to dashboard Cite

SummaryHumans engagement in music rests on underlying elements such as the listeners’ cultural background and general interest in music, all shaping the way music is processed in the brain and perceived. Crucially, these factors modulate how listeners anticipate musical events, a process inducing instantaneous neural responses as the music confronts these expectations. Measuring such neural correlates would represent a direct window into high-level brain processing of music. Here we recorded electroencephalographic and electrocorticographic brain responses as participants listened to Bach melodies. We assessed the relative contributions of the acoustic versus melodic components of the music to the neural signal. Acoustic features included envelope and its derivative. Melodic features included information on melodic progressions (pitch) and their tempo (onsets), which were extracted from a Markov model predicting the next note based on a corpus of Western music and the preceding proximal musical context. We related the music to brain activity with a linear temporal response function, and demonstrated that cortical responses to music encode melodic expectations. Specifically, individual-subject neural signals were better predicted by a combination of acoustic and melodic expectation features than by either alone. This effect was most pronounced at response latencies up to 350ms, and in both planum temporale and Heschl’s gyrus. Finally, expectations of pitch and onset-time of musical notes exerted independent cortical effects, and such influences were modulated by the listeners’ musical expertise. Overall, this study demonstrates how the interplay of experimental and theoretical approaches can yield novel insights into the cortical encoding of melodic expectations.

show abstract

“…It has been shown that other speech representations than the envelope, such as a spectrogram or phoneme representation, carry additional valuable information that can be beneficial for AAD (Brodbeck et al, 2018; December 6, 2018 DRAFT Broderick et al, 2018;Di Liberto et al, 2015). In this work, only broadband stimulus envelopes are used.…”

Section: B Future Improvementsmentioning

confidence: 99%

EEG-based detection of the locus of auditory attention with convolutional neural networks

Vandecappelle

Deckers

Das

et al. 2018

Preprint

View full text Add to dashboard Cite

When multiple people talk simultaneously, the healthy human auditory system is able to attend to one particular speaker of interest. Recently, it has been demonstrated that it is possible to infer to which speaker someone is attending by relating the neural activity, recorded by electroencephalography (EEG), with the speech signals. This is relevant for an effective noise suppression in hearing devices, in order to detect the target speaker in a multi-speaker scenario. Most auditory attention detection algorithms use a linear EEG decoder to reconstruct the attended stimulus envelope, which is then compared to the original stimuli envelopes to determine the attended speaker. Classifying attention within a short time interval remains the main challenge. We present two different convolutional neural network (CNN)-based approaches to solve this problem. One aims to select the attended speaker from a given set of individual speaker envelopes, and the other extracts the locus of auditory attention (left or right), without knowledge of the speech envelopes. Our results show that it is possible to decode attention within 1-2 seconds, with a median accuracy around 80%, without access to the speech envelopes. This is promising for neuro-steered noise suppression in hearing aids, which requires fast and accurate attention detection. Furthermore, the possibility of detecting the locus of auditory attention without access to the speech envelopes is promising for the scenarios in which per-speaker envelopes are unavailable. It will also enable establishing a fast and objective attention measure in future studies. Index TermsConvolutional neural networks (CNN), auditory attention detection (AAD), electroencephalography (EEG), neurosteered auditory prosthesis, brain-computer interface (BCI)

show abstract

Transformation from auditory to linguistic representations across auditory cortex is rapid and attention dependent for continuous speech

Cited by 8 publications

References 70 publications

Late cortical tracking of ignored speech facilitates neural selectivity in acoustically challenging conditions

Late cortical tracking of ignored speech facilitates neural selectivity in acoustically challenging conditions

Cortical encoding of melodic expectations in human temporal cortex

EEG-based detection of the locus of auditory attention with convolutional neural networks

Contact Info

Product

Resources

About