EEG-BASED SPEECH RECOGNITION - Impact of Temporal Effects

Abstract

Abstract:In this paper, we investigate the use of electroencephalograhic signals for the purpose of recognizing unspoken speech. The term unspoken speech refers to the process in which a subject imagines speaking a given word without moving any articulatory muscle or producing any audible sound. Early work by Wester (Wester, 2006) presented results which were initially interpreted to be related to brain activity patterns due to the imagination of pronouncing words. However, subsequent investigations lead to the hypothesis that the good recognition performance might instead have resulted from temporal correlated artifacts in the brainwaves since the words were presented in blocks. In order to further investigate this hypothesis, we run a study with 21 subjects, recording 16 EEG channels using a 128 cap montage. The vocabulary consists of 5 words, each of which is repeated 20 times during a recording session in order to train our HMM-based classifier. The words are presented in blockwise, sequential, and random order. We show that the block mode yields an average recognition rate of 45.50%, but it drops to chance level for all other modes. Our experiments suggest that temporal correlated artifacts were recognized instead of words in block recordings and back the above-mentioned hypothesis.

CNN Architectures and Feature Extraction Methods for EEG Imaginary Speech Recognition

“…One attempt at unspoken speech recognition was made by Marek Wester in 2006 [5] for his PhD thesis, with results that reached 50% accuracy in multiple class classification. However, the group later revealed in [6] that the experiment process favored the results because the signal acquisition protocol assumed to speak or think the exact stimulus multiple consecutive times, and this accidentally creates temporal correlation in EEG signals. This was an important discovery in data acquisition protocol for further created databases.…”

EEG-Based Silent Speech Interface and its Challenges: A Survey

“…BCI dataset [47] also used repetitions, but rather than using the beep sound, it applied a fixation cross shown from the monitor. The purpose of repetition block use was to maintain focus consistency [8]. Moreover, the longer the duration of the imagined speech, the easier for the subject to get sleepy because they performed the imagined speech state in silence.…”

“…The previous studies used different types of speech, such as vowels [26], phonemes [23], syllables [10], words [6]- [8], [16]- [18], [23], [26], [31], [32], sentences [7], and binary questions [20], [22]. The use of speech parts (vowels, phonemes, and syllables) mainly was to observe the brain when planning to produce the sound of words.…”

“…It represents natural communication, such as asking, responding, or making a statement [18], [32] and semantically differentiates homonyms [6], [16], [17], e.g., "two" and "to". Additionally, employing words with different lengths could help the model to differentiate words (such as "in" and "cooperate") [26] or repetition [8] affected the model and use words as commands for moving the cursor on the monitor ("up"/"down") [31]. Wester [7] also applied several words grouped based on their usability; digits (0-9), alpha corpus, phone-related words, MP3 player commands, Graduate Record Examination (GRE) Corpus, and Lecture Corpus.…”

“…least-squares CV [7] Visual 0-9; NATO phonetic alphabet (a-e); "yes", "no", etc. ; sentence 0.9-60 STFT+LDA HMM CV [8] Visual "alpha", "bravo", …, "echo" 1-300 DTCWT, LDA HMM CV [9] Audiovisual "/ba/", "/ku/" 3-18 Hilbert transform, spectral feature matched-filter CV [10] 3-20 spot of interest (SOI) LDA CV [11] 4-25 AR kNN CV [12] Visual "/a/", "/u/" 1-45 CSP SVM CV [13] Statistics, geometric mean, energy sum, entropy, wavelength LDA CV [14] 8-40 MFCC kNN/SVM CV [15] Audio "/aa/", "/ae/", "/l/", "/r/", "/m/", "/n/", "/uu/", "/ow/", "/s/", "/z/" 4-28 Spectrogram LDA CV [16] Visual "two", "to", "four", "for" N/A Voltage DT CV [17] Visual (Korean) 3 ("sam"), 5("oo"), 9 ("gu"), 10 ("sib"); cheek ("ppyam"), nose ("ko"), eye ("nun"), mouth ("ib") Audiovisual "/iy/", "/uw/", "/piy/", "/tiy/", "/diy/", "/m/", "/n/", "pat", "pot", "knew", "gnaw" 1-50 Statistics SVM CV [24] Statistics, MFCC, nonlinear features SVM CV [25] DWT DNN CV [26] Visual "/a/", "/i/", "/u/", "in", "out", "up", "cooperate", "independent" 8-70…”