Brain-computer interfaces (BCIs) have enabled individuals to control devices such as spellers, robotic arms, drones, and wheelchairs, but often these BCI applications are restricted to research laboratories. With the advent of virtual reality (VR) systems and the internet of things (IoT) we can couple these technologies to offer real-time control of a user’s virtual and physical environment. Likewise, BCI applications are often single-use with user’s having no control outside of the restrictions placed upon the applications at the time of creation. Therefore, there is a need to create a tool that allows users the flexibility to create and modularize aspects of BCI applications for control of IoT devices and VR environments. Using a popular video game engine, Unity, and coupling it with BCI2000, we can create diverse applications that give the end-user additional autonomy during the task at hand. We demonstrate the validity of controlling a Unity-based VR environment and several commercial IoT devices via direct neural interfacing processed through BCI2000.

No abstract

Accurate spatial co-registration of EEG electrode positions with individual head models is an important component for EEG source localization and imaging. Due to variations in head shape between individuals, this requires measurements of electrode locations in each individual. Existing hardware for digitization can be accurate, but also relatively expensive. With the goal of making digitization more accessible for a range of research laboratories, we have developed an open-source software tool that can make use of less expensive consumer virtual reality hardware for EEG electrode digitization. Here we describe our developed VRDigitizer system and compare it to existing digitization solutions. Experimental evaluations were performed in a phantom head model and in 12 human subjects. In our comparison experiments, VRDigitizer was able to measure electrode positions with a mean error of 3.74 mm, compared to 1.73 mm and 2.98 mm for the commercial systems tested.

Neural keyword spotting could form the basis of a speech brain-computer-interface for menu-navigation if it can be done with low latency and high specificity comparable to the “wake-word” functionality of modern voice-activated AI assistant technologies. This study investigated neural keyword spotting using motor representations of speech via invasively-recorded electrocorticographic signals as a proof-of-concept. Neural matched filters were created from monosyllabic consonant-vowel utterances: one keyword utterance, and 11 similar non-keyword utterances. These filters were used in an analog to the acoustic keyword spotting problem, applied for the first time to neural data. The filter templates were cross-correlated with the neural signal, capturing temporal dynamics of neural activation across cortical sites. Neural vocal activity detection (VAD) was used to identify utterance times and a discriminative classifier was used to determine if these utterances were the keyword or non-keyword speech. Model performance appeared to be highly related to electrode placement and spatial density. Vowel height (/a/ vs /i/) was poorly discriminated in recordings from sensorimotor cortex, but was highly discriminable using neural features from superior temporal gyrus during self-monitoring. The best performing neural keyword detection (5 keyword detections with two false-positives across 60 utterances) and neural VAD (100% sensitivity, ~1 false detection per 10 utterances) came from high-density (2 mm electrode diameter and 5 mm pitch) recordings from ventral sensorimotor cortex, suggesting the spatial fidelity and extent of high-density ECoG arrays may be sufficient for the purpose of speech brain-computer-interfaces.