Multimodal System for Training at Distance in a Virtual or Augmented Reality Environment for Users of Electric-Powered Wheelchairs

Borges, Ludymila Ribeiro; Martins, Felipe Roque; Naves, Eduardo Lázaro Martins; Bastos, Teodiano; Lucena, Vicente Ferreira de

doi:10.1016/j.ifacol.2016.11.146

Cited by 19 publications

(12 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent work has shown that EEG-based BCIs can successfully be used in combination with new technological developments such as AR to improve real-world practicality by offering a richer, more direct, and intuitive interface (Kansaku et al, 2010;Takano et al, 2011;Borges et al, 2016;Faller et al, 2017). However, very few fNIRS applications have explored this option (Afergan et al, 2015;McKendrick et al, 2016;Si-Mohammed et al, 2018;Hu et al, 2019).…”

Section: Using Ar In Bcis Offers a Great Flexibilitymentioning

confidence: 99%

An Augmented-Reality fNIRS-Based Brain-Computer Interface: A Proof-of-Concept Study

et al. 2020

View full text Add to dashboard Cite

Augmented reality (AR) enhances the user's environment by projecting virtual objects into the real world in real-time. Brain-computer interfaces (BCIs) are systems that enable users to control external devices with their brain signals. BCIs can exploit AR technology to interact with the physical and virtual world and to explore new ways of displaying feedback. This is important for users to perceive and regulate their brain activity or shape their communication intentions while operating in the physical world. In this study, twelve healthy participants were introduced to and asked to choose between two motorimagery tasks: mental drawing and interacting with a virtual cube. Participants first performed a functional localizer run, which was used to select a single fNIRS channel for decoding their intentions in eight subsequent choice-encoding runs. In each run participants were asked to select one choice of a six-item list. A rotating AR cube was displayed on a computer screen as the main stimulus, where each face of the cube was presented for 6 s and represented one choice of the six-item list. For five consecutive trials, participants were instructed to perform the motor-imagery task when the face of the cube that represented their choice was facing them (therewith temporally encoding the selected choice). In the end of each run, participants were provided with the decoded choice based on a joint analysis of all five trials. If the decoded choice was incorrect, an active error-correction procedure was applied by the participant. The choice list provided in each run was based on the decoded choice of the previous run. The experimental design allowed participants to navigate twice through a virtual menu that consisted of four levels if all choices were correctly decoded. Here we demonstrate for the first time that by using AR feedback and flexible choice encoding in form of search trees, we can increase the degrees of freedom of a BCI system. We also show that participants can successfully navigate through a nested menu and achieve a mean accuracy of 74% using a single motor-imagery task and a single fNIRS channel.

show abstract

Section: Using Ar In Bcis Offers a Great Flexibilitymentioning

confidence: 99%

An Augmented-Reality fNIRS-Based Brain-Computer Interface: A Proof-of-Concept Study

et al. 2020

View full text Add to dashboard Cite

show abstract

“…There is a necessity to offer customization of interfaces to new EPW users and EPW training [ 11 ]. However, a major challenge found in [ 10 ] was the integration of intelligent features with commercially available EPWs since they are proprietary systems, moreover, the cost of the technology can be a huge barrier.…”

Section: Introductionmentioning

confidence: 99%

“…This work is part of a joint research project between the Federal Universities of Espirito Santo (UFES), Uberlândia (UFU), and Amazonas (UFAM), Brazil [ 11 ]. This stage of the project is focused on the development of a safe training system and its technical validation.…”

Section: Introductionmentioning

confidence: 99%

Simulation System of Electric-Powered Wheelchairs for Training Purposes

Hernandez-Ossa

Montenegro-Couto

Longo

et al. 2020

Sensors

View full text Add to dashboard Cite

For some people with severe physical disabilities, the main assistive device to improve their independence and to enhance overall well-being is an electric-powered wheelchair (EPW). However, there is a necessity to offer users EPW training. In this work, the Simcadrom is introduced, which is a virtual reality simulator for EPW driving learning purposes, testing of driving skills and performance, and testing of input interfaces. This simulator uses a joystick as the main input interface, and a virtual reality head-mounted display. However, it can also be used with an eye-tracker device as an alternative input interface and a projector to display the virtual environment (VE). Sense of presence, and user experience questionnaires were implemented to evaluate this version of the Simcadrom in addition to some statistical tests for performance parameters like: total elapsed time, path following error, and total number of commands. A test protocol was proposed and, considering the overall results, the system proved to simulate, very realistically, the usability, kinematics, and dynamics of a real EPW in a VE. Most subjects were able to improve their EPW driving performance in the training session. Furthermore, all skills learned are feasible to be transferred to a real EPW.

show abstract

“…Second, compared to the motor imagery BCI paradigm, P300-based BCIs require less training time and achieve a higher accuracy and information transfer rate (amount of information sent per unit of time). Third, P300 allows for a larger number of possible commands [ 18 , 25 ], which can be used as the directional buttons (forward, up, down, right, right turn, left, and left turn) in the drone control system. Finally, the P300 paradigm provides an intuitive user interface (UI); what a user sees is what should be recognized [ 26 , 27 , 28 , 29 ], which is essential to control the drone easily.…”

Section: Introductionmentioning

confidence: 99%

P300 Brain–Computer Interface-Based Drone Control in Virtual and Augmented Reality

Kim

Lee

Kang

et al. 2021

Sensors

View full text Add to dashboard Cite

Since the emergence of head-mounted displays (HMDs), researchers have attempted to introduce virtual and augmented reality (VR, AR) in brain–computer interface (BCI) studies. However, there is a lack of studies that incorporate both AR and VR to compare the performance in the two environments. Therefore, it is necessary to develop a BCI application that can be used in both VR and AR to allow BCI performance to be compared in the two environments. In this study, we developed an opensource-based drone control application using P300-based BCI, which can be used in both VR and AR. Twenty healthy subjects participated in the experiment with this application. They were asked to control the drone in two environments and filled out questionnaires before and after the experiment. We found no significant (p > 0.05) difference in online performance (classification accuracy and amplitude/latency of P300 component) and user experience (satisfaction about time length, program, environment, interest, difficulty, immersion, and feeling of self-control) between VR and AR. This indicates that the P300 BCI paradigm is relatively reliable and may work well in various situations.

show abstract

Multimodal System for Training at Distance in a Virtual or Augmented Reality Environment for Users of Electric-Powered Wheelchairs

Cited by 19 publications

References 13 publications

An Augmented-Reality fNIRS-Based Brain-Computer Interface: A Proof-of-Concept Study

An Augmented-Reality fNIRS-Based Brain-Computer Interface: A Proof-of-Concept Study

Simulation System of Electric-Powered Wheelchairs for Training Purposes

P300 Brain–Computer Interface-Based Drone Control in Virtual and Augmented Reality

Contact Info

Product

Resources

About