Multiclass Classification of Word Imagination Speech With Hybrid Connectivity Features

Qureshi, Muhammad Naveed Iqbal; Min, Beomjun; Park, Hyeong-jun; Cho, Dongrae; Choi, Woosu; Lee, Boreom

doi:10.1109/tbme.2017.2786251

Cited by 63 publications

(36 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1) Large-set Decoding: Majority of works classify a closed-set vocabulary of units such as words [57], [58] and phrasal blocks [30]. This makes the scalability of the protocol to newer unseen test instances difficult.…”

Section: E Methodological Design Advantagesmentioning

confidence: 99%

Neural Speech Decoding During Audition, Imagination and Production

et al. 2020

View full text Add to dashboard Cite

Interpretation of neural signals to a form that is as intelligible as speech facilitates the development of communication mediums for the otherwise speech/motor-impaired individuals. Speech perception, production, and imagination often constitute phases of human communication. The primary goal of this paper is to analyze the similarity between these three phases by studying electroencephalogram(EEG) patterns across these modalities, in order to establish their usefulness for brain computer interfaces. Neural decoding of speech using such non-invasive techniques necessitates the optimal choice of signal analysis and translation protocols. By employing selection-by-exclusion based temporal modeling algorithms, we discover fundamental syllable-like units that reveal similar set of signal signatures across all the three phases. Significantly higher than chance accuracies are recorded for single trial multi-unit EEG classification using machine learning approaches over three datasets across 30 subjects. Repeatability and subject independence tests performed at every step of the analysis further strengthens the findings and holds promise for translating brain signals to speech non-invasively.

show abstract

Section: E Methodological Design Advantagesmentioning

confidence: 99%

Neural Speech Decoding During Audition, Imagination and Production

et al. 2020

View full text Add to dashboard Cite

show abstract

“…They also admit that the high performance is due to this characteristic of the paradigm. Nevertheless, it is implied that the classification performance of the imagined speech can be improved by selecting suitable features [28].…”

Section: A Feature Extractionmentioning

confidence: 99%

“…Six electrodes (F3, F4, C3, C4, P3, and P4) were used, and the reference and ground were placed over the left and right mastoids, respectively. In addition, some of the electrodes are located in Wernicke's area, which plays an important role in language processing [28], [43]. The classes consist of 5 Spanish vowels and 6 Spanish words.…”

Section: A Data Descriptionmentioning

confidence: 99%

Decoding Imagined Speech Based on Deep Metric Learning for Intuitive BCI Communication

Lee

2021

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

Imagined speech is a highly promising paradigm due to its intuitive application and multiclass scalability in the field of brain-computer interfaces. However, optimal feature extraction and classifiers have not yet been established. Furthermore, retraining still requires a large number of trials when new classes are added. The aim of this study is (i) to increase the classification performance for imagined speech and (ii) to apply a new class using a pretrained classifier with a small number of trials. We propose a novel framework based on deep metric learning that learns the distance by comparing the similarity between samples. We also applied the instantaneous frequency and spectral entropy used for speech signals to electroencephalography signals during imagined speech. The method was evaluated on two public datasets (6-class Coretto DB and 5-class BCI Competition DB). We achieved a 6-class accuracy of 45.00 ± 3.13% and a 5-class accuracy of 48.10 ± 3.68% using the proposed method, which significantly outperformed state-of-the-art methods. Additionally, we verified that the new class could be detected through incremental learning with a small number of trials. As a result, the average accuracy is 44.50 ± 0.26% for Coretto DB and 47.12 ± 0.27% for BCI Competition DB, which shows similar accuracy to baseline accuracy without incremental learning. Our results have shown that the accuracy can be greatly improved even with a small number of trials by selecting appropriate features from imagined speech. The proposed framework could be directly used to help construct an extensible intuitive communication system based on brain-computer interfaces.

show abstract

“…State-of-the-art neuroimaging and machine learning in computational neurosciences have offered novel strategies to study brain mechanisms [50][51][52][53]. This paper introduced a new framework which, when applied to multivariate high temporal resolution EEG, revealed microstate source generators and functional connectomics coordinated in speech perception.…”

Section: A Brain Dynamics In Speech Perceptionmentioning

confidence: 99%

Decoding Brain Dynamics in Speech Perception Based on EEG Microstates Decomposed by Multivariate Gaussian Hidden Markov Model

Duc

Lee

2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

This study aims to reveal dynamic brain networks during speech perception. All male subjects were presented five English vowel [a], [e], [i], [o], and [u] stimuli. Brain dynamics were decoded using multivariate Gaussian hidden Markov model (MGHMM), which trained on spatiotemporal patterns of broadband multivariate event-related potential amplitudes to identify distinct broadband EEG microstates (MS), microstate source imaging, and microstate functional connectivity (μFC). Obtained results showed fluctuated cortical generators and μFC in eight microstates throughout the perception. Microstate source imaging revealed involvements of bilateral (left-side dominance) posterior superior temporal cortex (TC), inferior frontal gyrus (IFG), and supramarginal regions in perception. Precentral cortex where primary motor cortex located was also significantly activated. These regions were early appeared at 96-151 ms (left-side dominance) and at 186-246 ms (left hemisphere only) after the stimuli onset. Results from μFC revealed significant increases in delta (2.5-4.5 Hz), theta (4.5-8.5 Hz), alpha (12.5-14.5 Hz), beta (22.5-24.5 Hz), low gamma (30.5-32.5, 38.5-40.5 Hz) but decreases in high gamma (42.5-46.5 Hz) bands in perception. Increased FC were observed mainly at; (1) microstate segments 34-95 ms (MS2) and 96-151 ms (MS3) in early stages, (2) microstate intervals 186-246 ms (MS5) and 297-449 ms (MS6) in subsequent stages of perception. We found that stronger statistical FC differences in perception at TCs, with respect to left IFG (Broca' area), left TC, and precentral areas. Furthermore, by conducting a comparative protocol measuring FC distinction degree, we showed performance improvements of 8.01% (p-value=0.0162), 14.41% (p-value=0.006) when compared MGHMM to well established Lehmann-based modified K-means, Atomize and Agglomerative Hierarchical Clustering and 8.791% (p-value=0.0097) over the combination of K-means and sliding window methods, respectively. This study indicates the usefulness of EEG microstates to investigate broadband brain dynamics in speech perception. The current findings based on male subjects would be generalized more by future studies with a larger appropriate sample size including female subjects.

show abstract

Multiclass Classification of Word Imagination Speech With Hybrid Connectivity Features

Abstract: In this study, electroencephalography data of imagined words were classified using four different feature extraction approaches. Eight subjects were recruited for the recording of imagination with five different words, namely; 'go', 'back', 'left', 'right', and 'stop'.

Cited by 63 publications

References 26 publications

Neural Speech Decoding During Audition, Imagination and Production

Neural Speech Decoding During Audition, Imagination and Production

Decoding Imagined Speech Based on Deep Metric Learning for Intuitive BCI Communication

Decoding Brain Dynamics in Speech Perception Based on EEG Microstates Decomposed by Multivariate Gaussian Hidden Markov Model

Contact Info

Product

Resources

About