Identification of vowels in consonant–vowel–consonant words from speech imagery based EEG signals

Abstract

Retrieval of unintelligible speech is a basic need for speech impaired and is under research for several decades. But retrieval of random words from thoughts needs a substantial and consistent approach. This work focuses on the preliminary steps of retrieving vowels from Electroencephalography (EEG) signals acquired while speaking and imagining of speaking a consonant-vowel-consonant (CVC) word. The process, referred to as Speech imagery is imagining of speaking to oneself silently in the mind. Speech imagery is a form of mental imagery. Brain connectivity estimators such as EEG coherence, Partial Directed Coherence, Directed Transfer Function and Transfer Entropy have been used to estimate the concurrency and causal dependence (direction and strength) between different brain regions. From brain connectivity results it has been observed that the left frontal and left temporal electrodes were activated for speech and speech imagery processes. These brain connectivity estimators have been used for training Recurrent Neural Networks (RNN) and Deep Belief Networks (DBN) for identifying the vowel from the subject's thought. Though the accuracy level was found to be varying for each vowel while speaking and imagining of speaking the CVC word, the overall classification accuracy was found to be 72% while using RNN whereas a classification accuracy of 80% was observed while using DBN. DBN was found to outperform RNN in both the speech and speech imagery processes. Thus, the combination of brain connectivity estimators and deep learning techniques appear to be effective in identifying the vowel from EEG signals of subjects' thought.

Recognition of EEG Signals from Imagined Vowels Using Deep Learning Methods

“…The intra-subject training model CNNeeg1-1 with BD1 database (M = 0.6562, SD = 0.0123) ( Figure 7 ) and database BD2 (M = 0.8566, SD = 0. 0446) ( Figure 15 ) in EEG imagined vowel recognition (/a/,/e/,/i/,/o/,/u/) had an accuracy comparable or superior to other works developed with DL for imagined vowel recognition (/a/,/e/,/i/,/o/,/u/) such as: DBN with an accuracy of 80% with 6 subjects [ 18 , 40 ] and an accuracy of 87. 96% with 3 subjects [ 18 ]; with RNN an accuracy of 70% with 6 subjects [ 40 ]; with CNN an accuracy of 32.75% with 15 subjects [ 41 , 42 ] and an accuracy of 35.68% with 15 subjects [ 42 ].…”

“…Additionally, to reduce the effect of the low signal to noise ratio of EEG signals, there are alternative DL methods using EEG signal preprocessing for imagined vowels, such as: filtering from 2 Hz to 40 Hz, artifact detection and removal with Independent Component Analysis (ICA), and analysis with Hessian approximation preconditioning; eigenvalues of the covariance matrix [ 18 ]; 50 Hz LPF-IIR low-pass filters, 0.5 Hz HPF-IIR high-pass filters, and feature vectors consisting of EEG coherence, partial directed coherence (PDC), Direct Transfer Function (DFT) and transfer entropy [ 40 ].…”

Decoding Covert Speech From EEG-A Comprehensive Review

“…In Chengaiyan et al ( 2020 ), 50 consonant-vowel-consonant words were used as the prompts. All the five vowels were considered and for each vowel, 10 words were used.…”

“…In Chengaiyan et al ( 2020 ), brain connectivity features such as coherence (Thatcher et al, 2004 ), partial directed coherence (PDC) (Sameshima and Baccalá, 1999 ), direct transfer function (DTF) (Kaminski and Blinowska, 1991 ), and transfer entropy (Schreiber, 2000 ) were computed for each band of the EEG signal. The EEG frequency bands considered were delta, theta, alpha, beta and gamma.…”

Possibility of long-termed prediction of NACPs and NADPs of COVID-19 in different countries and regions via tanh basic functions

“…Analysis and prediction of COVID-19 are investigated in some foreign countries in [ 10 – 12 ]. Prediction of NACP and the plateau phase of COVID-19 in China is investigated in [ 13 , 14 ]. Treatment and prognosis of COVID-19 are studied in [ 15 , 16 ].…”