Using EEG and Deep Learning to Predict Motion Sickness Under Wearing a Virtual Reality Device

Abstract

Virtual Reality (VR) research has been widely applied in many fields. VR promises to deliver the experience that is beyond the user's imagination. One of the advantages of VR is the feeling it gives of being there. VR can provide experiences impossible in the real world, such as flying, diving in deep water, exploring outer space, or living with dinosaurs. Despite the improvements in the software and hardware, the problem of motion sickness remains. We implement a deep learning model to train and predict motion sickness. A questionnaire is a well-known method to measure motion sickness. The weakness of the questionnaire is the measurement carried out after the user experiences motion sickness symptoms. By using the deep learning and EEG, the system will learn and classify motion sickness. The system learns the user's EEG pattern when they begin to feel the sickness symptoms. The system will be trained using deep learning to identify the sickness patterns in the future. By the EEG patterns, the system can predict the sickness symptoms before it occurs. Our model outperforms traditional models in loss values, accuracy, and F-measure metrics in Roller Coaster. With other datasets, our model also performs well. Our model can achieve 82.83% accuracy from the dataset. We also found that the time steps to predict motion sickness during 5 minute periods is a suitable configuration.

A Review of Deep Learning Approaches to EEG-Based Classification of Cybersickness in Virtual Reality

“…The final list of four papers [26][27][28][29] included in this review were closely scrutinized and encoded to capture a variety of attributes in relation to experimental procedures, data preprocessing steps, and DL architecture choices. Table 1 provides a detailed summary of these papers.…”

“…Lastly, one paper was a review paper reporting no empirical data. This screening resulted in a final list of four papers that meet the eligibility criteria [26][27][28][29]. Therefore, the current review focused on these four papers.…”

“…Given that such an attempt requires automatic detection of cybersickness on the basis of patterns observed in EEG signals, traditional machine learning algorithms (e.g., Support Vector Machines, Naïve Bayes, and k-Nearest Neighbors) have been extensively applied to the analysis of EEG data for automatic cybersickness detection [20][21][22][23][24][25]. Moreover, recent advances in deep learning (DL) algorithms, which are aimed at learning representations from data in successive layers of processing in neural networks, and increasing availability of computational resources for DL have paved the way for the application of DL frameworks to EEG-based detection of cybersickness [26][27][28][29].…”

Workplace Mental State Monitoring during VR-Based Training for Offshore Environment

“…Saghafian et al also suggested the need to measure realism with emotions within a stressful VR experience [ 38 ]. Motion sickness, disorientation and discomfort are also potential stressors when experiencing the VR [ 47 , 59 ]. Measuring these aspects of the VR using questionnaires has the limitation of inaccuracy as the user experiences are reported after the VR session [ 59 ].…”

“…Motion sickness, disorientation and discomfort are also potential stressors when experiencing the VR [ 47 , 59 ]. Measuring these aspects of the VR using questionnaires has the limitation of inaccuracy as the user experiences are reported after the VR session [ 59 ]. Using the EEG signals, Liao et al developed a prediction model to detect motion sickness in VR.…”