Generating EEG features from Acoustic features

Krishna, Gautam; Tran, Co; Carnahan, Mason; Han, Yan; Tewfik, A.H.

doi:10.23919/eusipco47968.2020.9287498

Cited by 9 publications

(21 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Generating EEG features from Acoustic features (Krishna et al 2020, EUSIPCO): In Krishna et al (2021a), the authors developed an RNN-based forward regression model. It can be seen as the inverse problem of the EEG-based synthesis from Krishna et al (2020).…”

Section: Recurrent Neural Networkmentioning

confidence: 99%

Relating EEG to continuous speech using deep neural networks: a review

Puffay¹,

Accou²,

Bollens³

et al. 2023

Preprint

View full text Add to dashboard Cite

Objective. When a person listens to continuous speech, a corresponding response is elicited in the brain and can be recorded using electroencephalography (EEG). Linear models are presently used to relate the EEG recording to the corresponding speech signal. The ability of linear models to find a mapping between these two signals is used as a measure of neural tracking of speech. Such models are limited as they assume linearity in the EEG-speech relationship, which omits the nonlinear dynamics of the brain. As an alternative, deep learning models have recently been used to relate EEG to continuous speech, especially in auditory attention decoding (AAD) and single-speech-source paradigms. Approach. This paper reviews and comments on deep-learning-based studies that relate EEG to continuous speech in AAD and single-speech-source paradigms. We point out recurrent methodological pitfalls and the need for a standard benchmark of model analysis. Main results. We gathered 28 studies. The main methodological issues we found are biased cross-validations, data leakage leading to over-fitted models, or disproportionate data size compared to the model's complexity. In addition, we address requirements for a standard benchmark model analysis, such as public datasets, common evaluation metrics, and good practices for the match-mismatch task. Significance. We are the first to present a review paper summarizing the main deep-learning-based studies that relate EEG to speech while addressing methodological pitfalls and important considerations for this newly expanding field. Our study is particularly relevant given the growing application of deep learning in EEG-speech decoding.

show abstract

Section: Recurrent Neural Networkmentioning

confidence: 99%

Relating EEG to continuous speech using deep neural networks: a review

Puffay¹,

Accou²,

Bollens³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…This may be the reason for the plenty of articles that have been published. Various features for both language and dialect recognition are available in the literature such as MFCC (see table (3)), singular values decomposition and Linear Predictive Codes (LPC). Koolagudi et al extracted several MFCC sets, which were different in dimension (6, 8, 13, 19, 21, 29 and 35), from the speech signal for identifying a language such as Bangla, Assamese, Guajarati, Hindi, Nepali, Kashmiri, Malayalam, Marathi, Rajasthani, Odia, Punjabi, Tamil, Telugu, Kannada, and Urdu.…”

Section: Language and Dialect Recognitionmentioning

confidence: 99%

“…The feature extraction is the process of extracting and tackling hidden information in the raw data signal [3]. Data can be more manageable by applying feature extraction because it removes all ineffective features from the data without losing any important or relevant data.…”

Section: Introductionmentioning

confidence: 99%

Mel Frequency Cepstral Coefficient and its Applications: A Review

2022

View full text Add to dashboard Cite

Feature extraction and representation has significant impact on the performance of any machine learning method. Mel Frequency Cepstrum Coefficient (MFCC) is designed to model features of audio signal and is widely used in various fields. This paper aims to review the applications that the MFCC is used for in addition to some issues that facing the MFCC computation and its impact on the model performance. These issues include the use of MFCC for non-acoustic signals, adopting the MFCC alone or combining it with other features, the use of time series versus global representation of the MFCC, following the standard form of the MFCC computation versus modifying its parameters, and supplying the traditional machine learning methods versus the deep learning methods..

show abstract

“…Further, Roy et al analyzed generated samples both qualitatively and quantitatively. Similarly, Krishna et al ( 2020 ) constructed a gated recurrent unit (GRU) (Chung et al, 2014 )-based generator and a GRU-based discriminator with the GAN loss function, i.e., Equation (1). Thus, Krishna et al augmented EEG data for speech recognition and achieved performance improvement.…”

Section: Advances In Data Augmentationmentioning

confidence: 99%

A Survey on Deep Learning-Based Short/Zero-Calibration Approaches for EEG-Based Brain–Computer Interfaces

Jeon

Jeong

et al. 2021

Front. Hum. Neurosci.

View full text Add to dashboard Cite

Brain–computer interfaces (BCIs) utilizing machine learning techniques are an emerging technology that enables a communication pathway between a user and an external system, such as a computer. Owing to its practicality, electroencephalography (EEG) is one of the most widely used measurements for BCI. However, EEG has complex patterns and EEG-based BCIs mostly involve a cost/time-consuming calibration phase; thus, acquiring sufficient EEG data is rarely possible. Recently, deep learning (DL) has had a theoretical/practical impact on BCI research because of its use in learning representations of complex patterns inherent in EEG. Moreover, algorithmic advances in DL facilitate short/zero-calibration in BCI, thereby suppressing the data acquisition phase. Those advancements include data augmentation (DA), increasing the number of training samples without acquiring additional data, and transfer learning (TL), taking advantage of representative knowledge obtained from one dataset to address the so-called data insufficiency problem in other datasets. In this study, we review DL-based short/zero-calibration methods for BCI. Further, we elaborate methodological/algorithmic trends, highlight intriguing approaches in the literature, and discuss directions for further research. In particular, we search for generative model-based and geometric manipulation-based DA methods. Additionally, we categorize TL techniques in DL-based BCIs into explicit and implicit methods. Our systematization reveals advances in the DA and TL methods. Among the studies reviewed herein, ~45% of DA studies used generative model-based techniques, whereas ~45% of TL studies used explicit knowledge transferring strategy. Moreover, based on our literature review, we recommend an appropriate DA strategy for DL-based BCIs and discuss trends of TLs used in DL-based BCIs.

show abstract

Generating EEG features from Acoustic features

Cited by 9 publications

References 14 publications

Relating EEG to continuous speech using deep neural networks: a review

Relating EEG to continuous speech using deep neural networks: a review

Mel Frequency Cepstral Coefficient and its Applications: A Review

A Survey on Deep Learning-Based Short/Zero-Calibration Approaches for EEG-Based Brain–Computer Interfaces

Contact Info

Product

Resources

About