Rapid P300 brain-computer interface communication with a head-mounted display

Abstract

Visual ERP (P300) based brain-computer interfaces (BCIs) allow for fast and reliable spelling and are intended as a muscle-independent communication channel for people with severe paralysis. However, they require the presentation of visual stimuli in the field of view of the user. A head-mounted display could allow convenient presentation of visual stimuli in situations, where mounting a conventional monitor might be difficult or not feasible (e.g., at a patient's bedside). To explore if similar accuracies can be achieved with a virtual reality (VR) headset compared to a conventional flat screen monitor, we conducted an experiment with 18 healthy participants. We also evaluated it with a person in the locked-in state (LIS) to verify that usage of the headset is possible for a severely paralyzed person. Healthy participants performed online spelling with three different display methods. In one condition a 5 × 5 letter matrix was presented on a conventional 22 inch TFT monitor. Two configurations of the VR headset were tested. In the first (glasses A), the same 5 × 5 matrix filled the field of view of the user. In the second (glasses B), single letters of the matrix filled the field of view of the user. The participant in the LIS tested the VR headset on three different occasions (glasses A condition only). For healthy participants, average online spelling accuracies were 94% (15.5 bits/min) using three flash sequences for spelling with the monitor and glasses A and 96% (16.2 bits/min) with glasses B. In one session, the participant in the LIS reached an online spelling accuracy of 100% (10 bits/min) using the glasses A condition. We also demonstrated that spelling with one flash sequence is possible with the VR headset for healthy users (mean: 32.1 bits/min, maximum reached by one user: 71.89 bits/min at 100% accuracy). We conclude that the VR headset allows for rapid P300 BCI communication in healthy users and may be a suitable display option for severely paralyzed persons.

A Comparison of Mobile VR Display Running on an Ordinary Smartphone With Standard PC Display for P300-BCI Stimulus Presentation

“…In spite of a higher ERP amplitude in VR, there were not significant differences in classification accuracy between the VR and PC conditions, which may be explained by the fact that the P200 component is stronger but also more variable in the VR condition ( Figure 6). This result is consistent with a previous study on HMDs [24].…”

“…In other words, proper ERPs are elicited using such an HMD, and the average classification accuracy is adequate. These results extend those of [24], which were gathered using the same kind of BCI but displayed on an expensive HMD. The data from the present study are freely available for download at https://doi.org/10.5281/zenodo.2605204.…”

“…The P200 is part of the long latency response, occurring after sensory responses and before high-level cognitive tasks such as the P300 [50]. It is involved in the cognitive process comparing sensory inputs with memory [51] and is modulated by arousal, attention [50], depth perception [52] or the intensity of the stimuli (see [24]).…”

“…Our results showed that the quality of EEG signal is similar under the two conditions. Previous studies [16], [21]- [24] conclude that performance of a VR BCI is equal to or better than that of a personal computer (PC) BCI. Recently, study [25] has shown significant improvement in BCI training when using a modern HMD and the motor imagery paradigm.…”

“…For example, [26] describes a system consisting of an HMD with an eyetracker, whereas [16] presents an immersive game with a CAVE. 1 Moreover, [16], [21], [24], [26], [27] describe BCI applications based on P300; [21], [23], [25], [28] applications with motor imagery; and [16], [21], [29] BCI systems based on SSVEP. References [24], [26] are studies combining an HMD and the P300 paradigm, as in the present study.…”