Neural Speech Decoding During Audition, Imagination and Production

Abstract

Interpretation of neural signals to a form that is as intelligible as speech facilitates the development of communication mediums for the otherwise speech/motor-impaired individuals. Speech perception, production, and imagination often constitute phases of human communication. The primary goal of this paper is to analyze the similarity between these three phases by studying electroencephalogram(EEG) patterns across these modalities, in order to establish their usefulness for brain computer interfaces. Neural decoding of speech using such non-invasive techniques necessitates the optimal choice of signal analysis and translation protocols. By employing selection-by-exclusion based temporal modeling algorithms, we discover fundamental syllable-like units that reveal similar set of signal signatures across all the three phases. Significantly higher than chance accuracies are recorded for single trial multi-unit EEG classification using machine learning approaches over three datasets across 30 subjects. Repeatability and subject independence tests performed at every step of the analysis further strengthens the findings and holds promise for translating brain signals to speech non-invasively.

Brain Computer Interface-EEG based Imagined Word Prediction Using Convolutional Neural Network Visual Stimuli for Speech Disability

“…There are many classi cation algorithms being applied in BCI technology. Researchers [13][14][15][16] reviewed the modern classi cation algorithms for data produced by an EEG device. The algorithms are classi ed into four main methods, namely adaptive classi ers, matrix and tensor classi ers, adaptive learning classi ers, and also deep learning classi ers [17][18][19].…”

Neural Decoding of Inferior Colliculus Multiunit Activity for Sound Category identification with temporal correlation and deep learning

“…They classified 3046 MEG signals using the ANN and CNN networks and obtained 96% accuracy for what was thought and 93% accuracy for what was spoken ([22]-Dash et al, 2020). Sharon et al, in their study entitled “Neural Speech Decoding During Audition, Imagination and Production”, achieved 54% success with the Gaussian mixture-based hidden Markov machine learning model from EEG signals collected from 30 individuals ([23]-Sharon et al., 2020). Heelan et al, conducted two experiments in their study entitled “Decoding speech from spike-based neural population records in secondary auditory cortex of non-human primates” in 2019.…”

Brain Computer Interface -EEG Based Imagined Word Prediction Using Convolutional Neural Network Visual Stimuli for Speech Disability

“…Researchers [13][14][15][16] reviewed the modern classification algorithms for data produced by an EEG device. The algorithms are classified into four main methods, namely adaptive classifiers, matrix and tensor classifiers, adaptive learning classifiers, and also deep learning classifiers [17][18][19].…”