Investigation of Delayed Response during Real-Time Cursor Control Using Electroencephalography

Abstract

Error-related brain activation has been investigated for advanced brain-machine interfaces (BMI). However, how a delayed response of cursor control in BMI systems should be handled is not clear. Therefore, the purpose of this study was to investigate how participants responded to delayed cursor control. Six subjects participated in the experiment and performed a wrist-bending task. For three distinct delay intervals (an interval where participants could not perceive the delay, an interval where participants could not be sure whether there was a delay or not, and an interval where participants could perceive the delay), we assessed two types of binary classifications (“Yes + No” vs. “I don’t know” and “Yes” vs. “No”) based on participants’ responses and applied delay times (thus, four types of classification, overall). For most participants, the “Yes vs. No” classification had higher accuracy than “Yes + No” vs. “I don’t know” classification. For the “Yes + No” vs. “I don’t know” classification, most participants displayed higher accuracy based on response classification than delay classification. Our results demonstrate that a class only for “I don’t know” largely contributed to these differences. Many independent components (ICs) that exhibited high accuracy in “Yes + No” vs. “I don’t know” response classification were associated with activation of areas from the frontal to parietal lobes, while many ICs that showed high accuracy in the “Yes vs. No” classification were associated with activation of an area ranging from the parietal to the occipital lobes and were more broadly localized in cortical regions than was seen for the “Yes + No” vs. “I don’t know” classification. Our results suggest that small and large delays in real-time cursor control differ not only in the magnitude of the delay but should be handled as distinct information in different ways and might involve differential processing in the brain.

Electroencephalography Reflects User Satisfaction in Controlling Robot Hand through Electromyographic Signals

“…Although delay appears to be a low-level concept, its effect is complex. Participants sometimes fail to perceive the correct delay [ 5 ]. Moreover, the awareness between certainty and uncertainty with respect to the delayed response of a robotic hand can induce a significant difference in the theta band of the parietal lobe [ 30 ].…”

“…For an individualized interface, the system must recognize the reactions of the user to its responses. Although some studies have investigated brain-related user responses, such as brain activity reflecting delayed response during real-time cursor control [ 5 ], brain activity during control for interfacing has rarely been investigated regarding subjective feelings. Brain activity related to a variety of subjective feelings has been investigated based on self-reported information.…”

Electroencephalogram Analysis Method to Detect Unspoken Answers to Questions Using Multistage Neural Networks

“…Moreover, direct recording of brain activity while imagining a word has not been reported in any previous fNIRS study. Conversely, EEG devices are generally easy to wear on a daily basis, and several studies have reported various approaches involving motor imagery EEG (MI-EEG) [5]- [10] and word imagery [11]- [15]. Support systems employing EEG-based BCI [2][5] [6][16] [17] can be used for communication, rehabilitation, and environmental system and electric wheelchair controls.…”

“…EEG signals have been analyzed to create BCI systems using analysis techniques such as EEG feature extraction from an event-related potential (ERP) response [11], the power spectrum and spectral centroid [13][17]- [19], Bayesian-based EEG feature extraction techniques [20], principal component analysis [21]- [24], independent component analysis (ICA) [15] [23][25]- [27], EEG pattern classifiers, artificial neural networks (ANN) [14][23] [28], k-nearest neighbor algorithm [29] [30], linear discriminant analysis (LDA) [15][31]- [33], SVM [12][13] [21][34]- [37], self-organizing maps [38], and fuzzy entropy [39]. Furthermore, over the past few years, DNN, its improved models [7][8][40]- [44], and convolutional neural networks (CNN) [9][10][45]- [51] have been employed for EEG feature extraction and pattern classification.…”

Brain Activity Reflects Subjective Response to Delayed Input When Using an Electromyography-Controlled Robot

“…Using questionnaires, several behavioral studies have investigated subjective evaluations of delayed output in specific situations such as first-person shooter games (Quax et al, 2004 ) and webpage loading (Guse et al, 2015 ). It was reported that in a real-time cursor control task, subtle and large delays in visual feedback were associated with different brain regions (Kim et al, 2020 ). Thus, a delay in the general human interface is a key factor that determines the subjective impression of the trustworthiness of the system, which also determines the user experience.…”