Detecting System Errors in Virtual Reality Using EEG Through Error-Related Potentials

Abstract

When persons interact with the environment and experience or witness an error (e.g. an unexpected event), a specific brain pattern, known as error-related potential (ErrP) can be observed in the electroencephalographic signals (EEG). Virtual Reality (VR) technology enables users to interact with computer-generated simulated environments and to provide multi-modal sensory feedback. Using VR systems can, however, be error-prone. In this paper, we investigate the presence of ErrPs when Virtual Reality users face 3 types of visualization errors: (Te) tracking errors when manipulating virtual objects, (Fe) feedback errors, and (Be) background anomalies. We conducted an experiment in which 15 participants were exposed to the 3 types of errors while performing a center-out pick and place task in virtual reality. The results showed that tracking errors generate error-related potentials, the other types of errors did not generate such discernible patterns. In addition, we show that it is possible to detect the ErrPs generated by tracking losses in single trial, with an accuracy of 85%. This constitutes a first step towards the automatic detection of error-related potentials in VR applications, paving the way to the design of adaptive and self-corrective VR/AR applications by exploiting information directly from the user's brain.

Customizing the human-avatar mapping based on EEG error related potentials

“…Holroyd and Coles [7] proposed that an error-processing system in the ACC serves as reinforcement-learning signals to correct errors. Further studies have also shown that error-related potentials (ErrPs) spontaneously arise when users experience BiEs during avatar-based interaction in VR [4,[8][9][10][11][12][13]. These findings support the notion of an accumulation of errors in these conditions [14][15][16], where cognitive processes in embodiment contribute to a global error in user experience.…”

A Structured Approach to Test the Signal Quality of Electroencephalography Measurements During Use of Head-Mounted Displays for Virtual Reality Applications

“…Unlike subjective measures, neurophysiological measures like the EEG directly reflect changes in cortical brain activity and therefore might provide, compared to subjective reports, a more direct basis for insights into the validity of theoretical frameworks that seek to explain psychophysical effects. It has already been shown that error-related potentials can indeed be detected while wearing an HMD, thus supporting self-adaptive VR environments (Si-Mohammed et al, 2020).…”

Neural sources of prediction errors detect unrealistic VR interactions

“…Midline cingulate EEG sources contributed to prediction error ERPs, 'PENs' , and may serve as a robust source to detect violations of user's predictions about the interaction with virtual worlds [7,8,20]. This source origin can be specifically and repeatedly probed for real-time BCI purposes, informing the technical system about the user's mental representation generating the predictions [20,58].…”

“…We and others have previously proposed the use of the frontal 'prediction error' negativity (PEN) as a feature for fast, real-time, detection of VR system errors which may, in turn, cause a loss in the sense of physical immersion [7][8][9]. Based on the idea that the brain has evolved to optimize motor behavior by detecting sensory mismatches, these studies promoted the usage of PENs to label a perceived loss in physical immersion, potentially impacting presence experience thereby providing continuous diagnostics about user experience.…”