Neural tracking of the speech envelope in cochlear implant users

Abstract

Objective: When listening to speech, the brain tracks the speech envelope. It is possible to reconstruct this envelope from EEG recordings. However, in people who hear using a cochlear implant (CI), the artifacts caused by electrical stimulation of the auditory nerve contaminate the EEG. This causes the decoder to produce an artifactdominated reconstruction, which does not reflect the neural signal processing. The objective of this study is to develop and validate a method for assessing the neural tracking of speech envelope in CI users.Approach: To obtain EEG recordings free of stimulus artifacts, the electrical stimulation is periodically interrupted. During these stimulation gaps, artifact-free EEG can be sampled and used to train a linear envelope decoder. Different recording conditions were used to characterize the artifacts and their influence on the envelope reconstruction.Main results: The present study demonstrates for the first time that neural tracking of the speech envelope can be measured in response to ongoing electrical stimulation. The responses were validated to be truly neural and not affected by stimulus artifact.Significance: Besides applications in audiology and neuroscience, the characterization and elimination of stimulus artifacts will enable future EEG studies involving continuous speech in CI users. Measures of neural tracking of the speech envelope reflect interesting properties of the listener's perception of speech, such as speech intelligibility or attentional state. Successful decoding of neural envelope tracking will open new possibilities to investigate the neural mechanisms of speech perception with a CI.

Cortical encoding of melodic expectations in human temporal cortex

“…1B) to quantify how well the acoustic (A) factors (e.g., signal envelope and its derivative) and melodic expectation or surprise (M) factors (e.g., pitch and onset-timing) can predict the EEG and ECoG responses to music (Crosse et al 2016). Since the prediction quality is considered to be an estimate of how strongly a stimulus property is encoded in the EEG data (Di Liberto et al 2015Brodbeck, Presacco, et al 2018;Somers et al 2018;Verschueren et al 2019), and since cortical signals are assumed to be modulated by the various A and M factors above, we consequently expected the combination of both the acoustic and surprise features to predict the neural responses better than either set of features alone (Fig. 1C).…”

Robust EEG-Based Decoding of Auditory Attention With High-RMS-Level Speech Segments in Noisy Conditions

“…The ability of selective attention of target streams from interferences is not only grounded in the acoustic properties of clean and noisy speech (e.g., spatial, spectral, and temporal cues), but also accounts for responses in any part of the central auditory pathway ( Snyder et al, 2012 ). Some researchers have investigated speech signal processing methods via the examination of neural responses to facilitate the attended speech recognition of hearing assistance devices in complex auditory scenes (e.g., Christensen et al, 2018 ; Miran et al, 2018 ; Somers et al, 2019 ). Several advantages could be derived from the incorporation of neural responses in speech signal processing.…”

“…Among these speech features, amplitude fluctuations of speech stimuli at low frequencies (i.e., the speech temporal envelope) have been used extensively as inputs for the decoding of auditory attention in online daily-life applications (e.g., Mirkovic et al, 2015 ; Van Eyndhoven et al, 2016 ; Christensen et al, 2018 ) employing non-invasive neuroimaging techniques (e.g., EEG). Use of the speech temporal envelope has enabled the achievement of high auditory attention decoding accuracy (e.g., Horton et al, 2014 ; Kong et al, 2014 ; Somers et al, 2019 ), as demonstrated by the reliability of cortical tracking (i.e., neural phase-locking) of attended speech at low brain oscillation frequencies (i.e., the delta and theta bands; Doelling et al, 2014 ). In envelope-based auditory attention decoding models, however, the cortical tracking of attended speech may be attenuated with decreased speech intelligibility, despite the lack of change in the speech temporal envelope ( Ding et al, 2014 ; Iotzov and Parra, 2019 ).…”

EEG-based diagnostics of the auditory system using cochlear implant electrodes as sensors

“…However, with the possibility to record continuous EEG, measures that correlate with higher-level aspects of hearing can be used to steer hearing devices. An EEG-based measure that predicts speech understanding [Vanthornhout et al, 2018;Lesenfants et al, 2019] was recently developed and has been shown to be measurable in CI users [Somers et al, 2018;Verschueren et al, 2019]. It is based on neural tracking of the speech envelope and can be measured from single-trial EEG recordings while listening to natural speech [Ding and Simon, 2013;Vanthornhout et al, 2018].…”

“…Recording of long-latency responses such as cortical evoked potentials (CEP) requires recording windows up to hundreds of milliseconds. Furthermore, auditory measures in response to continuously ongoing stimuli have been shown to relate to speech intelligibility, such as the auditory steady state response (ASSR) [Gransier et al, 2019] or speech envelope tracking responses [Ding and Simon, 2013;Vanthornhout et al, 2018;Somers et al, 2018;Lesenfants et al, 2019;Verschueren et al, 2019]. Measurement and analysis of these responses requires continuous EEG recordings.…”