A Ternary Hybrid EEG-NIRS Brain-Computer Interface for the Classification of Brain Activation Patterns during Mental Arithmetic, Motor Imagery, and Idle State

Abstract

The performance of a brain-computer interface (BCI) can be enhanced by simultaneously using two or more modalities to record brain activity, which is generally referred to as a hybrid BCI. To date, many BCI researchers have tried to implement a hybrid BCI system by combining electroencephalography (EEG) and functional near-infrared spectroscopy (NIRS) to improve the overall accuracy of binary classification. However, since hybrid EEG-NIRS BCI, which will be denoted by hBCI in this paper, has not been applied to ternary classification problems, paradigms and classification strategies appropriate for ternary classification using hBCI are not well investigated. Here we propose the use of an hBCI for the classification of three brain activation patterns elicited by mental arithmetic, motor imagery, and idle state, with the aim to elevate the information transfer rate (ITR) of hBCI by increasing the number of classes while minimizing the loss of accuracy. EEG electrodes were placed over the prefrontal cortex and the central cortex, and NIRS optodes were placed only on the forehead. The ternary classification problem was decomposed into three binary classification problems using the “one-versus-one” (OVO) classification strategy to apply the filter-bank common spatial patterns filter to EEG data. A 10 × 10-fold cross validation was performed using shrinkage linear discriminant analysis (sLDA) to evaluate the average classification accuracies for EEG-BCI, NIRS-BCI, and hBCI when the meta-classification method was adopted to enhance classification accuracy. The ternary classification accuracies for EEG-BCI, NIRS-BCI, and hBCI were 76.1 ± 12.8, 64.1 ± 9.7, and 82.2 ± 10.2%, respectively. The classification accuracy of the proposed hBCI was thus significantly higher than those of the other BCIs (p < 0.005). The average ITR for the proposed hBCI was calculated to be 4.70 ± 1.92 bits/minute, which was 34.3% higher than that reported for a previous binary hBCI study.

Brain-Based Binary Communication Using Spatiotemporal Features of fNIRS Responses

“…This would enable direct within-session feedback, which may heighten motivation in the participants and subsequently BCI performance Nijboer et al, 2010). Lastly, efforts to combine fNIRS with other modalities, such as EEG, have shown to improve classification accuracy significantly (Fazli et al, 2012;Zephaniah and Kim, 2014;Shin et al, 2018;Rezazadeh Sereshkeh et al, 2019) and are worth further investigation.…”

Performance Improvement of Near-Infrared Spectroscopy-Based Brain-Computer Interface Using Regularized Linear Discriminant Analysis Ensemble Classifier Based on Bootstrap Aggregating

“…Datasets denoted by "dataset I" and "dataset II" can be freely downloaded at: http://doc.ml.tu-berlin.de/hBCI/ (Shin et al, 2017b) and "dataset III" can be downloaded at: http://dx.doi.org/10.14279/depositonce-5830 (Shin et al, 2018c). "Dataset IV" is a NIRS dataset used in the study of Shin et al (2018b). All data processing was performed using MATLAB R2018b (Mathworks, MA, United States) and the BBCI toolbox 1 (Blankertz et al, 2016).…”

“…In the original articles (Shin et al, 2017b(Shin et al, , 2018b, the four datasets were preprocessed in different manners. For the sake of fair performance comparison, all datasets were preprocessed in the same manner.…”

“…Location of the NIRS channels for (A) datasets I-II, (B) dataset III, and (C) dataset IV. These figures were amended fromShin et al (2017b;2018b;2018c). Blue and red crosses are the specific positions of source and detector, respectively.…”

Effect of EOG Signal Filtering on the Removal of Ocular Artifacts and EEG‐Based Brain‐Computer Interface: A Comprehensive Study

“…The functional dynamics of the brain have been thoroughly investigated over the course of many years using noninvasive brain imaging techniques [1][2][3][4][5]. Electroencephalography (EEG), for example, is a portable neuroimaging system that can be used to assess different functional brain states [6][7][8][9].…”