Exploring Large Virtual Environments by Thoughts Using a Brain–Computer Interface Based on Motor Imagery and High-Level Commands

Abstract

Brain-computer interfaces (BCI) are interaction devices that enable users to send commands to a computer by using brain activity only. In this paper, we propose a new interaction technique to enable users to perform complex interaction tasks and to navigate within large virtual environments (VE) by using only a BCI based on imagined movements (motor imagery). This technique enables the user to send high-level mental commands, leaving the application in charge of most of the complex and tedious details of the interaction task. More precisely, it is based on points of interest and enables subjects to send only a few commands to the application in order to navigate from one point of interest to the other. Interestingly enough, the points of interest for a given VE can be generated automatically thanks to the processing of this VE geometry. As the navigation between two points of interest is also automatic, the proposed technique can be used to navigate efficiently by thoughts within any VE. The input of this interaction technique is a newly-designed self-paced BCI which enables the user to send three different commands based on motor imagery. This BCI is based on a fuzzy inference system with reject options. In order to evaluate the efficiency of the proposed interaction technique, we compared it with the state of the art method during a task of virtual museum exploration. The state of the art method uses low-level commands, which means that each mental state of the user is associated with a simple command such as turning left or moving forward in the VE. In contrast, our method based on high-level commands enables the user to simply select its destination, leaving the application performing the necessary movements to reach this destination. Our results showed that with our interaction technique, users can navigate within a virtual museum almost twice as fast as with low-level commands, and with nearly half the commands, meaning with less stress and more comfort for the user. This suggests that our technique enables efficient use of the limited capacity of current motor imagerybased BCI in order to perform complex interaction tasks in VE, opening the way to promising new applications.

Combining BCI with Virtual Reality: Towards New Applications and Improved BCI

“…OpenViBE 2 is a free software platform to design, test and use BCI [63]. OpenViBE has been successfully used for the three major families of BCI: Motor Imagery [46], P300 [10] and SSVEP [44]. Ogre3D 3 Fig.…”

“…Since both OpenViBE and Ogre3D have VRPN support, either natively or through contributions, they are able to communicate efficiently in order to design BCI-based VR applications. Those softwares have been used to design BCI-based VR applications such as those described in [46,44,47]. Generally not only VRPN, but any other interface (like proprietary TCP, UDP connections) can be used to communicate with existing VR systems.…”

“…In order to alleviate the fatigue caused by BCI-based navigation in VR, as well as to efficiently use the small number of MI tasks recognized by a BCI, INRIA 7 also proposed a new interaction technique for BCI-based VR applications [46]. This technique, based on a 3-class self-paced BCI, provides the user with high-level commands, thus leaving the application in charge of performing the complex and tedious details (low-level aspects) of the interaction task.…”

BCI-Based Navigation in Virtual and Real Environments

“…One more work that classifies three MI classes is [32]; MI tasks are not directly interpreted as navigation commands but they are used to move through a decision tree in order to choose among several high level options.…”

Development of a MATLAB-based toolbox for brain computer interface applications in virtual reality

“…Tn this work, for improving the training period of subject, virtual reality (VR)-based feedback is displayed to the subject. Tn recent years some researchers have applied VR in a BCI based on Ml [9], [10]. In this work the authors developed a MATLAB-based toolbox, because MA TLAB software is a widely available numerical computing and visualization environment.…”