Detecting Learning Stages within a Sensor-Based Mixed Reality Learning Environment Using Deep Learning

Ogunseiju, Omobolanle; Akinniyi, Abiola; Gonsalves, Nihar; Khalid, Mohammad; Akanmu, Abiola

doi:10.1061/jccee5.cpeng-5169

Cited by 2 publications

(1 citation statement)

References 76 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Long Short-Term Memory network, a variant of recurrent neural network, can learn long-term dependencies between time steps of data and predict future time-series sequences of the data. Long Short-Term Memory (LSTM) network has been used for sequential learning tasks like construction equipment activity analysis (Hernandez et al, 2019), construction workers' safety harness usage (Guo et al, 2023), mixed reality learning environment (Ogunseiju et al, 2023) and, fatigue detection and early warning system (Liu et al, 2020) that need historical timeseries data in the decision-making process. Therefore, this study investigates the extent to which workers' mental workload due to exoskeleton-use can be predicted from EEG data using Long Short-Term Memory network.…”

Section: Introductionmentioning

confidence: 99%

Predicting Mental Workload of Using Exoskeletons for Construction Work: A Deep Learning Approach

Akanmu,

Afolabi,

Okunola

2023

CONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality

Self Cite

View full text Add to dashboard Cite

Exoskeletons are gaining attention as a potential solution for addressing low back injury in the construction industry. However, use of active back-support exoskeletons in construction can trigger unintended consequences which could increase mental workload of users while working with exoskeletons. Prolonged increase in mental workload could impact workers’ wellbeing and productivity. Prediction of mental workload during exoskeleton-use could inform strategies to mitigate the triggers. This study investigates a machine-learning framework for predicting mental workload of workers while using active back-support exoskeletons for construction work. Laboratory experiments were conducted wherein Electroencephalography (EEG) data were collected from participants wearing active back-support exoskeletons to perform flooring task. The EEG data underwent preprocessing, including band filtering, notch filtering, and independent component analysis, to remove artifacts and ensure data quality. A regression-based Long Short-Term Memory network was trained to forecast future time steps of the processed EEG data. The performance of the network was evaluated using root mean square error (RMSE) and r-squared (R2). A RMSE of 0.1527 and R2 of 0.9665 indicating good fit and strong correlation, respectively, were observed between the predicted and actual EEG data. Results of the comparison between the actual and predicted mental workload also show strong correction with an R2 of 0.8692. The findings motivate research directions into real-time monitoring of mental workload of workers during exoskeleton-use. The study has significant implications for stakeholders, enabling them to gain a deeper understanding of the impact of mental workload while using exoskeletons thereby providing opportunities for mitigation

show abstract