Decoding Auditory EEG Responses Using an Adapted Wavenet

This paper describes the auditory EEG challenge, organized as one of the Signal Processing Grand Challenges at ICASSP 2023. The challenge provides EEG recordings of 85 subjects who listened to continuous speech, as audiobooks or podcasts, while their brain activity was recorded. EEG recordings of 71 subjects were provided as a training set such that challenge participants could train their models on a relatively large dataset. The remaining 14 subjects were used as held-out subjects in evaluating the challenge. The challenge consists of two tasks that relate electroencephalogram (EEG) signals to the presented speech stimulus. The first task, match-mismatch, aims to determine which of two speech segments induced a given EEG segment. In the second regression task, the goal is to reconstruct the speech envelope from the EEG. For the match-mismatch task, the performance of different teams was close to the baseline model, and the models did generalize well to unseen subjects. In contrast, For the regression task, the top teams significantly improved over the baseline models in the held-out stories test set while failing to generalize to unseen subjects.

show abstract

Sea-Wave: Speech Envelope Reconstruction From Auditory EEG With an Adapted WaveNet

Yang,

Van Dyck,

Hulle

2024

IEEE Open J. Signal Process.

View full text Add to dashboard Cite

Speech envelope reconstruction from EEG is shown to bear clinical potential to assess speech intelligibility. Linear models are commonly used to this end, but they have recently been outperformed in reconstruction scores by non-linear deep neural networks, particularly by dilated convolutional networks. This study presents Sea-Wave, a WaveNet-based architecture for speech envelopes reconstruction that outperforms the state-of-the-art model. Our model is an extension of our submission for the Auditory EEG Challenge of the ICASSP Signal Processing Grand Challenge 2023. We improve upon our prior work by evaluating model components and hyperparameters through an ablation study and hyperparameter search, respectively. Our best subject-independent model achieves a Pearson correlation of 22.58% on seen and 11.58% on unseen subjects. After subject-specific fine-tuning, we find an average relative improvement of 30% for the seen subjects and a Pearson correlation of 56.57% for the best seen subject. Finally, we explore several model visualizations to obtain a better understanding of the model, the differences across subjects and the EEG features that relate to auditory perception.

show abstract

Decoding Auditory EEG Responses Using an Adapted Wavenet

Cited by 4 publications

References 1 publication

Cross-Attention-Guided Wavenet for Mel Spectrogram Reconstruction in The ICASSP 2024 Auditory EEG Challenge

Cross-Attention-Guided Wavenet for Mel Spectrogram Reconstruction in The ICASSP 2024 Auditory EEG Challenge

Auditory EEG Decoding Challenge for ICASSP 2023

Sea-Wave: Speech Envelope Reconstruction From Auditory EEG With an Adapted WaveNet

Contact Info

Product

Resources

About