Estimating Cognitive Workload in an Interactive Virtual Reality Environment Using EEG

Tremmel, Christoph; Herff, Christian; Satô, Tetsuya; Rechowicz, K; Yamani, Yusuke; Krusienski, Dean J.

doi:10.3389/fnhum.2019.00401

Cited by 93 publications

(64 citation statements)

References 82 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The output layer consists of 1 node only, which classifies whether the output belongs to a sick or normal state. The equation for the LSTM with a forget gate is shown by equation (9). "forgetting" unimportant information.…”

Section: Proposed Methodologymentioning

confidence: 99%

“…An EEG can be used to measure the cognitive learning which takes place in the VR world. Using an EEG, the system can actively measure the cognitive state of the user, unlike the Brain Computer Interface (BCI) which is commonly used to measure the cognitive state [9]. Use of an EEG with VR enables study of language processing in a naturalistic environment [10].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Using EEG and Deep Learning to Predict Motion Sickness Under Wearing a Virtual Reality Device

et al. 2020

View full text Add to dashboard Cite

Virtual Reality (VR) research has been widely applied in many fields. VR promises to deliver the experience that is beyond the user's imagination. One of the advantages of VR is the feeling it gives of being there. VR can provide experiences impossible in the real world, such as flying, diving in deep water, exploring outer space, or living with dinosaurs. Despite the improvements in the software and hardware, the problem of motion sickness remains. We implement a deep learning model to train and predict motion sickness. A questionnaire is a well-known method to measure motion sickness. The weakness of the questionnaire is the measurement carried out after the user experiences motion sickness symptoms. By using the deep learning and EEG, the system will learn and classify motion sickness. The system learns the user's EEG pattern when they begin to feel the sickness symptoms. The system will be trained using deep learning to identify the sickness patterns in the future. By the EEG patterns, the system can predict the sickness symptoms before it occurs. Our model outperforms traditional models in loss values, accuracy, and F-measure metrics in Roller Coaster. With other datasets, our model also performs well. Our model can achieve 82.83% accuracy from the dataset. We also found that the time steps to predict motion sickness during 5 minute periods is a suitable configuration.

show abstract

Section: Proposed Methodologymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Using EEG and Deep Learning to Predict Motion Sickness Under Wearing a Virtual Reality Device

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Virtual/augmented reality (V/AR) technologies together with BCI could offer immersive experiences and have many potential applications including arts and neurofeedback (Andujar et al, 2015 ; Tremmel et al, 2019 ; Putze et al, 2020 ). Brain painting allows a user to draw lines in a virtual canvas by brain signals, which gives an alternative communication channel for people with paretic motor functions (Botrel et al, 2015 ).…”

Section: Neuroplasticity Sensors Signal Processing Modeling Anmentioning

confidence: 99%

“…McClinton et al have developed a brain painting application using VR environment (McClinton et al, 2019 ). Another work has evinced VR-BCI to measure cognitive workload that can contribute to neuroergonomics (Tremmel et al, 2019 ). Studies have also used immersive VR as a better neurofeedback option as compared to the computer screen leading to increased BCI accuracy (Luu et al, 2016 ; Škola et al, 2019 ; Vourvopoulos et al, 2019 ; Juliano et al, 2020 ).…”

Section: Neuroplasticity Sensors Signal Processing Modeling Anmentioning

confidence: 99%

Progress in Brain Computer Interface: Challenges and Opportunities

Saha

Mamun

Ahmed

et al. 2021

Front. Syst. Neurosci.

227

View full text Add to dashboard Cite

Brain computer interfaces (BCI) provide a direct communication link between the brain and a computer or other external devices. They offer an extended degree of freedom either by strengthening or by substituting human peripheral working capacity and have potential applications in various fields such as rehabilitation, affective computing, robotics, gaming, and neuroscience. Significant research efforts on a global scale have delivered common platforms for technology standardization and help tackle highly complex and non-linear brain dynamics and related feature extraction and classification challenges. Time-variant psycho-neurophysiological fluctuations and their impact on brain signals impose another challenge for BCI researchers to transform the technology from laboratory experiments to plug-and-play daily life. This review summarizes state-of-the-art progress in the BCI field over the last decades and highlights critical challenges.

show abstract

“…For example, Thapa N et al used EEG signals to study VR-based training programs, which improved the cognitive and physical functions of MCI patients compared to ordinary training [19]. Tremmel et al adjusted users' interaction in the virtual environment based on the average spectral amplitude difference of EEG signals recorded in the use of VR devices [20]. However, the current research of cognitive abilities based on EEG focuses mainly on cognitive training of participants with cognitive impairments.…”

Section: Introductionmentioning

confidence: 99%

Measuring the Impacts of Virtual Reality Games on Cognitive Ability Using EEG Signals and Game Performance Data

Wan

Wang

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Gamification has been widely used in training and enhancing cognitive ability, such as working memory and attention. For example, 3D games can enhance participants' working memory, thereby improving their learning efficiency, and the powerful immersion of VR games can improve the attention of patients with ADHD. However, previous works have not elaborated to what extent the two different game modes can affect cognitive ability respectively, nor have they determined which game mode has a greater effect on the cognitive level. Exploring the impact of different game modes on cognitive ability can help better apply gamification to the training and improvement of cognitive ability, and quantifying cognitive ability through human physiological signals such as brain electrical activity level (EEG) can help evaluate the changes of cognitive ability in different game modes more accurately. Therefore, different from previous studies, we used EEG signals and game performance data (such as game scores and time to complete the game) to calculate participants' cognitive level quantitatively, and did comparative experiments to study cognitive abilities in different game modes and the temporal characteristics of cognitive ability changes. We also explored whether gender would affect cognitive ability. Our research has the following findings. First, compared with the 3D mode, participants get better game scores and higher EEG scores in the VR mode, the time to complete the game task is shorter, and the time to reach the best state of working memory is shorter. Second, participants' gender has no significant effect on working memory, but there is a significant difference in attention in the VR game mode, it takes less time for males to reach the peak of attention than for females. Our conclusion indicates that VR games have greater potential in the field of training and enhancing cognitive ability than 3D games.

show abstract

Estimating Cognitive Workload in an Interactive Virtual Reality Environment Using EEG

Cited by 93 publications

References 82 publications

Using EEG and Deep Learning to Predict Motion Sickness Under Wearing a Virtual Reality Device

Using EEG and Deep Learning to Predict Motion Sickness Under Wearing a Virtual Reality Device

Progress in Brain Computer Interface: Challenges and Opportunities

Measuring the Impacts of Virtual Reality Games on Cognitive Ability Using EEG Signals and Game Performance Data

Contact Info

Product

Resources

About