Detecting System Errors in Virtual Reality Using EEG Through Error-Related Potentials

Si-Mohammed, Hakim; Lopes-Dias, Catarina; Duarte, Maria Eduarda; Argelaguet, Ferran; Jeunet, Camille; Casiez, Géry; Müller-Putz, Gernot; Lécuyer, Anatole; Scherer, Reinhold

doi:10.1109/vr46266.2020.1581262194646

Cited by 7 publications

(10 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other studies have pursued similar machine-learning approaches, with ERP correlates of movement error attribution to oneself or another VR agent yielding a single-trial classification accuracy of 73% (Dimova-Edeleva et al, 2022), and ERP correlates of visuospatial tracking errors reaching a singletrial classification accuracy of 85% (Si-Mohammed et al, 2020). In the study by Si-Mohammed and colleagues (2020), two other types of VR errors (erroneous feedback and visual anomalies in the background) could not reliably be detected at the level of the users' EEG.…”

Section: Discussionmentioning

confidence: 99%

Using event-related brain potentials to evaluate motor-auditory latencies in virtual reality

Feder¹,

Miksch²,

Grimm³

et al. 2023

Preprint

View full text Add to dashboard Cite

Actions in the real world have immediate sensory consequences. Mimicking these in digital environments is within reach, but technical constraints usually impose a certain latency (delay) between user actions and system responses. It is important to assess the impact of this latency on the users, ideally with measurement techniques that do not interfere with their digital experience. One such unobtrusive technique is electroencephalography (EEG), which can capture the users’ brain activity associated with motor responses and sensory events by extracting event-related potentials (ERPs) from the continuous EEG recording. Here we exploit the fact that the amplitude of sensory ERP components (specifically, N1 and P2) reflects the degree to which the sensory event was perceived as an expected consequence of an own action (self-generation effect). Participants (N=24) elicit auditory events in a virtual-reality (VR) setting by entering codes on virtual keypads to open doors. In a within-participant design, the delay between user input and sound presentation is manipulated across blocks. Occasionally, the virtual keypad is operated by a simulated robot instead, yielding a control condition with externally generated sounds. Results show that N1 (but not P2) amplitude is reduced for self-generated relative to externally generated sounds, and P2 (but not N1) amplitude is modulated by delay of sound presentation in a graded manner. This dissociation between N1 and P2 effects maps back to basic research on self-generation of sounds. We suggest P2 amplitude as a candidate read-out to assess the quality and immersiveness of digital environments with respect to system latency.

show abstract

Section: Discussionmentioning

confidence: 99%

Using event-related brain potentials to evaluate motor-auditory latencies in virtual reality

Feder¹,

Miksch²,

Grimm³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…They could show that the EEG responded to increases in task load that were not detectable from purely behavioral data. Si-Mohammed et al [29] detect and differentiate another class of interaction obstacles from EEG, namely visualization errors in VR, such as tracking errors or background anomalies. They show that by exploiting so called error potentials, tracking errors can be detected robustly, while other types of errors could not be identified.…”

Section: Related Workmentioning

confidence: 99%

Multimodal Differentiation of Obstacles in Repeated Adaptive Human-Computer Interactions

Putze

Salous

2021

26th International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

Human Computer Interaction can be impeded by various interaction obstacles, impacting a user's perception or cognition. In this work, we detect and discriminate such interaction obstacles from different data modalities to compensate for them through User Interface (UI) adaptation. For example, we detect memory-based obstacles from brain activity and compensate through repetition of information in the UI; we detect visual obstacles from user behavior and compensate by complementing visual with auditory information in the UI. Online cognitive adaptive systems should be able to decide the most suitable UI adaptation given inputs from several obstacles detectors. In this paper, we employ a Bayesian fusion approach upon different underlying obstacles detectors over multiple consecutive interaction sessions. Experimental results show that the model promisingly outperforms the baseline in the first interaction with an average accuracy of 72.5% and further improves drastically in subsequent interactions with additional information, with an average accuracy of 98%. CCS CONCEPTS• Human-centered computing → User models.

show abstract

“…Therefore, ErrPs can be used to improve BCIs' performance either in a corrective manner, by allowing corrective actions, or in an adaptive manner, by reducing the possibility of future errors [7][8][9][10][11]. The real-time detection of ErrPs is pertinent in BCIs used by persons with motor impairments and also in applications targeting healthy users [12][13][14][15]. The incorporation of ErrPs' detection in a BCI promotes a smoother interaction with its user.…”

Section: Introductionmentioning

confidence: 99%

Online asynchronous detection of error-related potentials in participants with a spinal cord injury using a generic classifier

Lopes-Dias,

Sburlea,

Breitegger

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

A brain-computer interface (BCI) user awareness of an error is associated with a cortical signature named error-related potential (ErrP). The incorporation of ErrPs' detection in BCIs can improve BCIs' performance.Objective: This work is three-folded. First, we investigate if an ErrP classifier is transferable from able-bodied participants to participants with spinal cord injury (SCI). Second, we test this generic ErrP classifier with SCI and control participants, in an online experiment without offline calibration. Third, we investigate the morphology of ErrPs in both groups of participants.Approach: We used previously recorded electroencephalographic (EEG) data from able-bodied participants to train an ErrP classifier. We tested the classifier asynchronously, in an online experiment with 16 new participants: 8 participants with SCI and 8 able-bodied control participants. The experiment had no offline calibration and participants received feedback regarding the ErrPs' detection from its start. For a matter of fluidity of the experiment, the feedback regarding false positive ErrP detections was not presented to the participants but these detections were taken into account in the evaluation of the classifier. The generic classifier was not trained with the user's brain signals. Still, its performance was optimized during the online experiment with the use of personalized decision thresholds. The classifier's performance was evaluated using trial-based metrics, which consider the asynchronous detection of ErrPs during the entire trials' duration.Main results: Participants with SCI presented a non-homogenous ErrP morphology, and four of them did not present clear ErrP signals. The generic classifier performed above chance level in participants with clear ErrP signals, independently of the SCI (11 out of 16 participants). Three out of the five participants that obtained chance level results with the generic classifier would have not benefited from the use of a personalized classifier.Significance: This work shows the feasibility of transferring an ErrP classifier from able-bodied participants to participants with SCI, for asynchronous detection of ErrPs in an online experiment without offline calibration, which provided immediate feedback to the users.

show abstract

Detecting System Errors in Virtual Reality Using EEG Through Error-Related Potentials

Cited by 7 publications

References 35 publications

Using event-related brain potentials to evaluate motor-auditory latencies in virtual reality

Using event-related brain potentials to evaluate motor-auditory latencies in virtual reality

Multimodal Differentiation of Obstacles in Repeated Adaptive Human-Computer Interactions

Online asynchronous detection of error-related potentials in participants with a spinal cord injury using a generic classifier

Contact Info

Product

Resources

About