Prediction and detection of virtual reality induced cybersickness: a spiking neural network approach using spatiotemporal EEG brain data and heart rate variability

Abstract

Virtual Reality (VR) allows users to interact with 3D immersive environments and has the potential to be a key technology across many domain applications, including access to a future metaverse. Yet, consumer adoption of VR technology is limited by cybersickness (CS)—a debilitating sensation accompanied by a cluster of symptoms, including nausea, oculomotor issues and dizziness. A leading problem is the lack of automated objective tools to predict or detect CS in individuals, which can then be used for resistance training, timely warning systems or clinical intervention. This paper explores the spatiotemporal brain dynamics and heart rate variability involved in cybersickness and uses this information to both predict and detect CS episodes. The present study applies deep learning of EEG in a spiking neural network (SNN) architecture to predict CS prior to using VR (85.9%, F7) and detect it (76.6%, FP1, Cz). ECG-derived sympathetic heart rate variability (HRV) parameters can be used for both prediction (74.2%) and detection (72.6%) but at a lower accuracy than EEG. Multimodal data fusion of EEG and sympathetic HRV does not change this accuracy compared to ECG alone. The study found that Cz (premotor and supplementary motor cortex) and O2 (primary visual cortex) are key hubs in functionally connected networks associated with both CS events and susceptibility to CS. F7 is also suggested as a key area involved in integrating information and implementing responses to incongruent environments that induce cybersickness. Consequently, Cz, O2 and F7 are presented here as promising targets for intervention.

A machine learning approach to evaluate the impact of virtual balance/cognitive training on fall risk in older women

“…The researchers concluded that while various models for detecting cybersickness have been developed, there is no model to predict early adverse events in VEs. Future, more accurate and effective ML approaches will undoubtedly be inspired by current knowledge of how the brain works, as well as the new brain/organism models/approaches being developed and linking them to these bio-digital frontiers ( Benelli et al, 2023 ; Daşdemir, 2023 ; Yang et al, 2023 ). These and similar studies ( Chang et al, 2023 ; Souchet et al, 2023 ) indicate, as does our research, that we are still at the beginning of understanding the potentially beneficial and detrimental effects of digital worlds on their users, especially older adults ( Drazich et al, 2023 ; Séba et al, 2023 ).…”

Predicting Tacit Coordination Success Using Electroencephalogram Trajectories: The Impact of Task Difficulty

“…Recent studies (e.g., [31,32]) have explored innovative approaches in EEG analysis and machine learning, such as Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM) and the Spiking Neural Network (SNN). While these methods offer valuable insights, they are particularly tailored to addressing specific methodological challenges, such as time series analysis and noise robustness.…”

“…The EEG data underwent several preprocessing steps, which were successfully implemented in previous studies (e.g., [3,8]). These steps included applying band-pass filtering (BPF) in the range of [1,32] Hz to capture the delta, theta, alpha, and beta frequency bands. These bands are crucial for the cognitive tasks under investigation and are less susceptible to high-frequency noise [21,22].…”